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1  .  EXECUTIVE  SUMMARY 

In  fiscal  year  1969  a  DARPA  program  entitled  "Resource 
Sharing  Computer  Networks"  was  initiated.  The  research  carried 
out  under  this  program  has  since  become  internationally  famous  as 
the  ARPANET. 

This  DARPA  program  has  created  no  less  than  a  revolution  in 
computer  technology  and  has  been  one  of  the  most  successful 
projects  ever  undertaken  by  DARPA?  The  program  has  initiated 
extensive  changes  in  the  Defense  Department's  use  of  computers  as 
well  as  in  the  use  of  computers  by  the  entire  public  and  private 
sectors,  both  in  the  United  States  and  around  the  worlu.  Just  as 
the  telephone,  the  telegraph,  and  the  printing  press  had 
far-reaching  effects  on  human  intercommunication,  the  widespread 
utilization  of  computer  networks  which  has  been  catalyzed  by  the 
ARPANET  project  represents  a  similarly  far-reaching  change  in  the 
use  of  computers  by  mankind.  The  full  impact  of  the  technical 
changes  set  in  motion  by  this  project  may  not  be  understood  for 
many  years. 

In  1975  the  ARPANET  was  successfully  transferred  to  the 
Defense  Communications  Agency  which  has  operated  it  since  that 
time. 

*  Defense  Advanced  Research  Projects  Agency  (DARPA) 
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1.  PROGRAM  OBJECTIVE  AND  TECHNICAL  NEED 
1.1  Defense  Program  Addressed 

The  DARPA  program  had  the  following  objectives:  (1)  To 
develop  techniques  and  obtain  experience  on  interconnecting 
computers  in  such  a  way  that  a  very  broad  class  of  interactions 
are  possible,  and  (2)  To  improve  and  increase  computer  research 
productivity  through  resource  sharing.  It  was  envisioned  that  by 
establishing  a  network  tying  computer  research  centers  together, 
both  goals  would  be  achieved.  In  fact,  the  most  efficient  way  to 
develop  the  techniques  needed  for  an  effective  network  was 
thought  to  be  by  involving  the  research  talent  at  these  centers 
in  prototype  activity.  Just  as  time-shared  computer  systems 
permitted  groups  of  hundreds  of  individual  users  to  share 
hardware  and  software  resources  with  one  another,  it  was  thought 
that  networks  connecting  dozens  of  such  systems  would  permit 
resource  sharing  between  thousands  of  users.  Each  system,  by 
virtue  of  being  time-shared,  could  offer  any  of  its  services  to 
another  computer  system  on  demand.  The  most  important  criterion 
for  the  type  of  network  interconnection  desired  was  that  any  user 
or  program  on  any  of  the  networked  computers  be  able  to  utilize 
any  program  or  subsystem  available  on  any  other  computer  without 
having  to  modify  the  remote  program. 
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This  objective  was  an  entirely  new  and  different  approach  to 
an  extremely  serious  problem  which  existed  throughout  botn  the 
Defense  Department  and  society  at  large.  The  many  hundreds  of 
computer  centers  in  the  Defense  Department  and  the  many  other 
thousands  of  computer  centers  in  the  private  and  public  sectors 
operate  almost  completely  autonomously.  Each  computer  center  is 
forced  to  recreate  all  the  software  and  data  files  that  it  wishes 
to  utilize.  In  many  cases  this  involves  the  complete 
reprogramming  of  software  or  reformatting  of  data  files.  This 
duplication  and  redundant  effort  is  extremely  costly  and  time 
consuming.  In  fiscal  year  1969  DARPA  estimated  that  such 
duplicative  efforts  more  than  double  the  national  costs  of 
creating  and  maintaining  the  software.  There  had  been  other 
completely  different  attempts  to  address  this  problem,  such  as 
attempts  at  language  standards  for  computers,  attempts  at 
standardizing  the  types  of  hardware,  and  attempts  at  automatic 
translation  between  computer  languages.  Although  each  such 
approach  had  some  value  and  utility,  the  problems  of  trying  to 
share  computer  software  resources  cr  files  was  truly  enormous. 

In  addition  to  the  general  problem  shared  with  the  rest  of 
the  scientific  community,  the  Defense  Department  also  faces 
certain  special  problems  having  to  do  with  training.  Military 
personnel  trained  to  use  one  manufacturer's  equipment  must  often 
be  trained  again  to  use  another's.  Machines  procured  from 
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different  manufacturers  require  as  many  different  user  training 
programs  as  there  are  machines,  thus  inhibiting  positive  transfer 
of  training  that  could  accumulate  through  the  rotation  of 
military  personnel.  Those  data  files  and  programs  which  have 
common  utility  to  many  military  organisations  and  installations 
must  be  stored,  created  and  maintained  separately  at  each 
different  machine.  Military  systems  interconnected  in  a 
distributed  interactive  network  obviate  such  constraints. 

Another  objective  of  th<  program  was  to  permit  the  linking 
of  specialized  computers  to  the  many  general  purpose  computer 
centers.  It  was  thought  that  with  the  then  recent  improvements 
in  the  hardware  area,  it  would  become  nost  cost  effective  to 
design  and  construct  computers  efficient  at  specialized  tasks 
(e.g.,  compiling,  list  processing  and  information  retrieval). 
Making  such  machines  available  to  all  the  computer  research 
establishments  would  significantly  increase  the  capability  at 
these  other  centers. 

This  program  was  addressed  at  no  less  then  changing  the  use 
of  computers  by  the  entire  Defense  Department.  It  was  clea.ly 
intended  that  the  use  of  such  a  computer  network  would  permit 
resource  sharing  within  and  across  the  military  services  and 
throughout  the  Defense  research  community. 
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1 .2  State  of  the  Art  at  Program  Inception 

By  the  date  of  the  program  plan  in  the  late  1960s  most  of 
the  specific  technologies  required  for  a  computer  network  had 
individually  been  achieved  in  some  form.  For  example,  there  had 
been  many  connections  of  phone  lines  to  computers  (e.g.,  the  SAGE 
system,  Air  Line  reservations  systems,  and  time-sharing  systems). 
However,  there  had  been  only  a  very  small  number  of  attempts  to 
connect  computers  together  for  the  purpose  of  experimenting  with 
the  sharing  of  resources. 

o  In  the  early  1960s  an  attempt  was  made  to  link  computers 
together  at  the  Western  Data  Processing  Center  at  UCLA 
for  the  purpose  of  enabling  similar  computers  to  perform 
load  sharing.  A  similar  experiment  was  also  performed 
at  Bell  Laboratories  and  achieved  reasonable  success  for 
several  years. 

o  A  number  of  networks  were  constructed  for  the  primary 

t 

purpose  of  message  handling,  including  a  Westinghouse 
inventory  control  system  and  several  air  line 
reservations  networks.  The  SITA  network  for  air  line 
reservations  was  surprisingly  advanced  in  concept  in  the 
mid-1960s,  but  details  about  SITA  were  generally  not 
known  in  the  U.S.  computing  community.  In  any  case,  the 
techniques  used  for  such  message  systems  were  special 
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purpose  in  nature  and  were  not  readily  transferable  into 
the  general  area  of  inter-computer  communications. 

o  A  direct  progenitor  of  the  ARPANET  was  an  effort  made  in 
the  mid-1960s  to  achieve  a  coupling  between  academic 
computing  expertise  and  the  operation  of  the  SDC  Q32 
computer.  This  effort  led  to  a  phone  line  connection 
between  the  Q32  at  SDC  and  the  TX2  at  Lincoln 
Laboratory,  and  demonstrated  the  relative  ease  of 
modifying  time  sharing  systems  to  permit  network 
interactions . 

Aside  from  the  technical  problems  of  interconnecting 
computers  with  communications  circuits,  the  notion  of  computer 
networks  had  been  considered  in  a  number  of  places  from  a 
theoretical  point  of  view.  Of  particular  note  was  work  done  by 
Paul  Baran  and  others  at  the  Rand  Corporation  in  a  study  "On 
Distributed  Communications"  in  the  early  1960's.  Also  of  note 
was  work  done  by  Donald  Davies  and  others  at  the  National 
Physical  Laboratory  in  England  in  the  mid-1960's. 

In  sum,  at  the  time  of  the  initiation  of  the  ARPANET  program 
in  fiscal  year  1969  many  of  the  requisite  ideas  had  been 
considered  and  many  of  the  requisite  technical  bits  and  pieces 
had  been  attempted  in  some  form,  but  no  significant  attempt  had 
ever  been  made  to  put  them  together  into  a  resource  sharing 
computer  network. 
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1.3  Specific  Technological  Problems  Addressed 

The  technological  problems  of  building  the  ARPANET  can  be 
considered  at  many  different  levels  of  detail.  At  the  top  level, 
there  were  really  two  problems: 

1.  To  construct  a  "subnetwork"  consisting  of  telephone 
circuits  and  switching  nodes  whose  reliability,  delay 
characteristics,  capacity,  and  cost  would  facilitate 
resource  sharing  among  the  computers  on  the  network. 

2.  To  understand,  design,  and  implement  the  protocols  and 
procedures  within  the  operating  systems  of  each 
connected  computer,  in  order  to  allow  the  use  of  the  new 
subnetwork  by  those  computers  in  sharing  resources. 

Within  these  two  major  technological  problems,  there  were, 
of  course,  a  large  number  of  sub-problems;  including  the 
engineering  of  the  phone  circuit  connections,  the  topology  of  the 
network,  the  selection  of  switching  node  equipment,  the  design  of 
line  disciplines  to  work  through  phone  line  errors,  the  routing 
problem,  and  many  others. 

1 .4  Expected  Payoff/Time-Frame/Costs 

The  goals  for  the  ARPANET  project  were  very  broad  and 
envisaged  a  significant  eventual  impact  on  the  use  of  computers 
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in  both  the  public  and  private  sectors.  However,  in  addition  to 
these  long  range  goals,  DARPA  visualized  some  quite  specific 
initial  payoff  in  the  form  of  improved  productivity  of  the  DARPA 
research  program  itself,  and  a  resulting  cost/performance  benefit 
to  the  services  from  DARPA  research.  In  fiscal  year  1969,  a 
number  of  computer  research  centers  throughout  the  country  were 
supported  in  whole  or  in  part  by  DARPA* s  Information  Processing 
Techniques  Office  (IPTO).  The  installation  of  an  effective 
network  tying  these  locations  together  would  substantially  reduce 
duplication  and  improve  the  transfer  of  scientific  results,  as 
well  as  develop  the  network  techniques  needed  by  the  military. 
The  research  output  of  these  projects  was  important  to  all  three 
Services  and  it  was  expected  that  this  output  could  be 
substantially  increased  for  the  same  dollar  cost  if  a  portion  of 
the  funds  were  utilized  for  the  network. 

In  addition,  initial  payoff  was  anticipated  in  the  form  of 
technology  transfer  from  the  ARPANET  project  in  three  ways: 

o  By  dissemination  of  new  scientific  knowledge  through 
conferences  and  the  appropriate  literature. 

o  By  transfer  of  management  of  the  ARPANET  to  a  common 
carrier,  and  the  resulting  availability  of  ARPANET 
services  to  other  groups  (such  as  Office  of  Education 
Regional  Laboratories,  NSF-supported  universities,  and 
various  user  groups  supported  by  the  NIH). 
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By  adoption  of  the  network  technology  by  specific 
military  groups  (such  as  the  National  Military  Command 
System  Support  Center  and  other  military  centers 
affiliated  »nth  it;  e.g.,  CINCPAC,  CINCEUR,  and  MACV). 
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2.  PROGRAM  DESCRIPTION  AND  EVOLUTION 
2.1  Program  Structure 

With  the  initiation  of  the  ARPANET  program  plan  in  early 
fiscal  year  1969,  DARPA  began  work  in  earnest  on  three  parallel 
paths  of  effort:  (1)  to  obtain  the  network  circuits;  (2)  to 
select  the  system  contractor  for  the  switching  nodes  and  the 
overall  design;  and  (3)  to  initiate  efforts  within  the  DARPA 
research  community  for  resource  sharing  experiments  and 
specialized  network  support. 

DECCO  was  able  to  handle  all  the  contractual  details  with 
the  common  carriers  for  circuit  leases.  Most  of  the  required  50 
kilobit  circuits  used  in  the  ARPANET  were  leased  through  DECCO 
from  AT&T,  but  a  small  number  of  circuits  were  leased  from  other 
carriers  such  as  General  Telephone.  In  addition,  DARPA  arranged 
for  a  special  point  of  contact  in  AT&T  (long  lines),  which 
greatly  facilitated  the  interactions  between  the  network  system 
contractor,  DARPA,  and  AT&T.  The  selection  of  network  node 
locations  and  the  intercede  connections  (and,  therefore,  the 
location  of  circuit  terminations)  was  a  specialized  topology 
problem  and  represented  a  difficult  theoretical  problem  in  its 
own  right.  To  help  solve  this  particular  problem,  DARPA 
contracted  with  the  Network  Analysis  Corporation  (NAC).  NAC  had 
developed  certain  networking  analysis  tools  and  via  this  DARPA 
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support,  such  tools  were  refined;  NAC’s  advice  on  topology  was 
sought  through  the  various  stages  of  ARPANET  growth. 

A  competitive  procurement  was  planned  for  the  selection  of 
the  contractor  to  design  the  switching  nodes  and  act  as  general 
systems  contractor.  An  RFP  was  prepared  and  issued  in  July  1968. 
Bolt  Beranek  and  Newman  Inc.  (BBN)  was  selected  as  the  systems 
contractor  for  the  design  of  the  subnetwork  and  the  switching 
nodes.  BBN  subcontracted  with  HoneyweXl  for  the  switching  node 
hardware  itself.  Over  the  life  of  the  ARPANET  program  from 
January  1969  until  the  transfer  to  DCA  in  July  of  1975,  BBN 
served  as  the  systems  contractor  for  the  design,  implementation, 
operation  and  maintenance  of  the  subnetwork;  BBN  is  still 
serving  that  role  under  the  aegis  of  the  Defense  Communications 
Agency  (DCA). 

The  research  groups  receiving  DARPA  IPTO  support  then  began 
considering  the  design  and  implementation  of  protocols  and 

procedures  and,  in  turn,  computer  program  modifications,  in  the 

I 

various  host  computers  in  order  to  use  the  subnetwork.  Several 
specific  responsibilities  were  arranged:  UCLA  was  specifically 
tasked  to  develop  and  run  a  "Network  Measurement  Center"  with  the 
objective  of  determining  the  performance  of  the  network;  SRI  was 
specifically  tasked  to  develop  and  operate  a  "Network  Information 
Center"  with  the  objective  of  collecting  information  about  the 
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network,  about  host  resources,  and  at  the  same  time  generating 
computer  based  tools  for  storing  and  accessing  that  collected 
information.  Beyond  these  two  specific  contracts,  some  rather  ad 
hoc  mechanisms  were  pursued  to  reach  agreement  between  the 
various  research  contractors  about  the  appropriate  "host 
protocols"  for  intercommunicating  over  the  subnetwork.  The 
"Network  Working  Group"  of  interested  individuals  from  the 
various  host  sites  was  rather  informally  encouraged  by  DARPA. 
After  a  time,  this  Network  Working  Group  became  the  forum  for, 
and  eventually  a  semi-official  approval  authority  for,  the 
discussion  of  and  issuance  of  host  protocols  to  be  implemented  by 
the  various  research  contractors.  Progress  in  this  area  was 
rather  slow  for  a  while,  but  with  time,  this  mechanism  eventually 
was  successful  in  establishing  effective  host  protocols. 

2,2  Major  Technical  Problems  and  Approaches 

In  a  program  of  this  duration  and  complexity,  it  is  not 
difficult  to  identify  many  dozens  of  important  technical  problems 
and  approaches  to  those  problems.  We  here  list  a  few  of  the 
problems  which  were  most  technically  challenging  in  the  early  few 
years  of  the  ARPANET  program.  A  few  additional  major  technical 
problems  will  be  listed  in  the  next  section  on  "Major  Changes  and 
Objectives". 
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o  TOPOLOGY  -  For  any  network  of  this  type  with  even  a 
dozen  nodes,  an  obvious,  early  recognized,  and  quite 
formidable  problem  is  topological  optimization. 
Assuming  that  the  node  locations  are  known,  the  number 
of  ways  of  arranging  M  links  among  N  nodes  is  very- 
large;  the  links  are  usually  available  in  discrete  sizes 
(bandwidth);  and  the  component  cost  structures,  time 
delay  functions,  and  reliability  functions  are  all 
typically  non-linear.  The  design  may  be  subject  to  many 
constraints,  including  maximum  or  average  time  delay, 
average  or  peak  throughput  requirements,  and  reliability 
requirements.  The  usual  goal  of  the  optimization  is  to 
provide  a  network  design  that  meets  all  constraints  at 
the  lowest  cost.  The  approach  to  this  problem  was  to 
design  an  elaborate  computer  program  to  assist  in  the 
optimization.  It  is  not  possible  to  use  an  exhaustive 
approach,  and  instead  the  approach  used  was  to  generate 
a  "starting  network"  and  then  to  perform  local 
transformations  to  the  topology  in  order  to  reach  a 
locally  optimum  network.  If  this  procedure  is  repeated 
with  many  starting  networks  and  if  the  resulting  locally 
optimum  networks  are  evaluated,  it  is  possible  to  find  a 
feasible  solution  with  costs  that  are  close  to  optimal. 
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o  ERROR  CONTROL  -  A  critical  necessity  for  a  resource 
sharing  computer  network  was  to  provide  reliable 
communication  and  one  component  of  such  reliability  was 
an  ability  to  work  through  the  expected  phone  line 
errors  on  the  50  kilobit  circuits.  The  approach  taken 
was  to  design  special  checksumming  hardware  at  the 
transmitting  and  receiving  end  of  each  50  kilobit 
circuit  in  the  network.  As  part  of  the  switching  node 
transmission  procedure,  a  powerful  24  bit  cyclic 

checksum  is  appended  to  every  packet  of  information  to 
be  transmitted.  Then,  at  the  receiving  node,  the 
checksum  is  recomputed  in  hardware  and  compared  with  the 


transmitted 

checksum. 

If 

there 

is 

no 

error , 

an 

acknowledgment 

(with  its 

own 

checksum) 

is 

sent 

back 

to 

the  transmitting  node. 

If 

there 

is 

an 

error , 

no 

acknowledgment 

is  sent 

and 

the 

packet 

will 

be 

retransmitted. 

o  HOST  INTERFACE  -  An  important  issue  was  the  proper 
design  of  an  interface  between  the  switching  node  and 
host  computers  of  many  different  types.  It  is  important 
to  allow  a  logical  match  between  the  switching  node 
computer  word  length  and  the  varying  word  lengths  of  the 
host  computers,  and  also  to  allow  the  input/output 
routines  of  the  switching  node  and  the  input/output 
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routines  of  the  host  computer  to  service  the  interface 
"cooperatively”  without  placing  an  undue  processing 
burden  or  tight  timing  constraints  on  either  machine. 
The  approach  taken  was  to  design  a  bit-serial  interface 
which  could  logically  stop  after  any  bit,  and  to  then 
require  that  each  host  computer  build  (obtain)  a 
specialized  small  hardware  unit  called  a  "special  host 
interface"  which  would  be  installed  in  the  host  machine 
itself  to  serve  the  network  connection.  Thus,  a  logical 
and  electrical  convention  was  specified  by  the  switching 
network  in  a  manner  intended  to  minimize  overall  trouble 
to  all  hosts,  then  each  host  was  required  to  meet  that 
standard  both  in  hardware  and  software. 

o  SWITCHING  NODE  PERFORMANCE  -  A  central  goal  of  the 
network  was  to  provide  resource  sharing  between  remote 
installations  in  such  a  way  that  a  local  user  could 
employ  a  remote  resource  without  degradation.  In 

particular,  this  required  that  the  subnetwork  be 
sufficiently  reliable,  have  sufficiently  low  delay,  have 
sufficiently  great  capacity  and  have  sufficiently  low 
cost,  that  remote  use  would  be  "as  attractive"  as  local 
use.  These  objectives  translated  into  a  major  technical 
problem  for  the  switching  node  itself  to  provide  low 
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delay  and  high  capacity  at  modest  cost*  The  approach 
taken  was  to  select  a  mini  computer  for  the  switching 
nodes  whose  I/O  system  was  very  efficient  and  to  write 
very  carefully  tailored  programs  in  machine  language  to 
optimize  the  capacity  and  the  low  delay  of  the  data  path 
in  the  switching  node.  Great  attention  wa3  paid  to 
minimizing  the  operating  time  of  the  inner  loops  of 
these  programs. 

o  REMOTE  CONTROL  -  A  special  and  new  problem  was  posed  by 
the  need  to  put  dozens  of  small  identical  mini  computers 
in  the  field,  in  an  environment  where  the  host 
connections  to  those  computers  was  somewhat  experimental 
and  where  the  programs  in  the  mini  computers  themselves 
were  under  development  and  were  changing  from  month  to 
month.  It  was  important  to  be  able  to  do  debugging  of 
software,  debugging  of  new  host  connections,  program 
modifications,  and  the  installation  of  new  programs 
without  the  costs  and  difficulties  associated  with 
having  manned  sites.  The  approach  taken  was  to  develop 
an  entirely  new  technology  of  remote  computer 

management.  A  Network  Control  Center  was  established 
for  the  subnetwork  and  software  was  developed  which  made 
it  possible  to  examine  or  change  the  operating  software 
in  any  node  of  the  net  from  the  central  network  control 
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center.  This  approach  made  it  possible  to  issue 
complete  new  versions  of  the  software  to  each  node  in 
the  network  from  a  central  place  in  an  hour  or  twc.  In 
addition,  each  node  reports  the  state  of  its  health  to 
the  central  place  periodically  and  provides  information 
on  which  to  base  debugging  and  maintenance  activities. 

o  ROUTING  -  In  a  non-fully  connected  network,  an  important 
problem  is  the  decision  process  by  which  each  node 
decides  how  to  route  information  to  reach  any  particular 
destination.  This  is  a  difficult  theoretical  problem 
and  there  are  many  different  approaches,  including  fixed 
routing,  random  routing,  centrally  controlled  routing, 
and  various  forms  of  distributed  adaptive  routing.  The 
approach  taken  was  to  use  a  distributed  adaptive  traffic 
routing  algorithm  which  estimates  on  the  basis  of 
information  from  adjacent  nodes  the  globally  proper 
instantaneous  path  for  each  message  in  the  face  of 
varying  input  traffic  loads  and  local  line  or  nod-? 
failures*  Each  IMP  keeps  tables  containing  an  estimate 
of  the  output  circuit  corresponding  to  the  minimum  delay 
path  to  each  other  IMP,  and  a  corresponding  estimate  of 
the  delay.  Periodically,  each  IMP  sends  its  current 
routing  estimates  to  its  neighbors;  whenever  an  IMP 
receives  such  a  message  it  updates  its  internal 
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estimates.  Thus,  information  about  changing  conditions 
is  regularly  propagated  throughout  the  network,  and 
whenever  a  packet  of  traffic  must  be  placed  on  the  queue 
for  an  output  circuit,  the  IMP  uses  its  latest  estimate 
of  the  best  path. 

4 

o  HOST  PROTOCOL  -  In  many  ways  a  computer  network  is  to 
host  computers  as  the  telephone  system  is  to  human 
users:  a  transparent  communication  medium  in  which  even 
after  the  caller  has  learned  how  to  insert  dimes  and 
dial,  it  is  still  necessary  that  he  speak  the  same 
language  as  the  person  called  in  order  for  useful 
communication  to  occur.  The  common  language  is  referred 
to  as  host  protocol,  and  the  problem  is  to  design  a  host 
protocol  which  is  sufficiently  powerful  for  the  kinds  of 
communication  that  will  occur  and  yet  can  be  implemented 

t 

in  all  of  the  various  different  host  computer  systems. 
The  initial  approach  taken  involved  the  development  of  a 
piece  of  software  called  a  "Network  Control  Program" 
which  would  reside  in  a  host  computer,  such  that 
processes  within  a  host  would  communicate  with  the 
network  through  this  Network  Control  Program.  The 
primary  function  of  the  NCP  is  to  establish  connections, 
break  connections,  switch  connections,  and  control  flow. 
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A  "layered”  approach  was  taken  such  that  more  complex 
procedures  (such  as  File  Transfer  Procedures)  were  built 
on  top  of  simpler  procedures  in  the  host  Network  Control 
Program. 

2.3  Major  Changes  in  Objectives  and  Approaches 

The  ARPANET  development  was  an  extremely  intense  activity  in 
which  contributions  were  made  by  many  of  the  best  computer 
scientists  in  the  United  States.  Thus,  almost  all  of  the  "major 
technical  problems"  already  mentioned  received  continuing 
attention  and  the  detailed  approach  to  those  problems  changed 
several  times  during  the  early  years  of  the  ARPANET  effort. 
However,  in  addition  several  more  major  changes  in  objectives 
were  introduced  once  the  initial  network  became  operational: 

o  THE  TIP  -  The  initial  nodal  switching  units,  called 
IMPs,  were  intended  to  interconnect  computers  and  high 
bandwidth  phone  lines.  At  the  outset  all  terminal 
access  to  the  network  was  via  terminal  connections  to 
the  hosts  themselves.  After  a  time  it  became  clear  that 
there  was  a  population  cf  users  for  which  terminal 
access  to  the  network  was  very  desirable,  but  who  were 
not  conveniently  able  to  access  the  network  via  a  host 
computer.  Thus,  a  new  nodal  switching  unit,  a  Terminal 
Interface  Message  Processor,  or  TIP,  was  defined  to 
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serve  the  purpose  of  an  IMP  plus  an  additional  function 
of  direct  terminal  access.  This  shift  resulted  in  the 
design  of  a  TIP  which  really  was  a  tiny  host  embedded  in 
a  switching  node  itself  and  permitted  the  direct 
connection  of  up  to  63  asynchronous  character-oriented 
terminals  to  the  switching  node.  The  TIP  became  the 
nodal  switching  unit  of  choice,  often  even  where  there 
was  a  local  host  computer;  this  allowed  connection  of 
both  hosts  and  terminals  at  that  location  directly  to 
the  network. 
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3.  SCIENTIFIC  AND  TECHNICAL  RESULTS  AND  ACCOMPLISHMENTS 

3.1  Results  of  the  Effort  in  Relation  to  the  Program  Objectives 

In  some  cases  a  major  program  can  be  seen  to  reach  its 
objectives  in  a  single  instant:  a  mushroom  cloud  at  Alamogordo, 
or  Armstrong  stepping  onto  the  moon.  In  other  cases,  like  the 
ARPANET,  an  equally  important  objective  may  be  reached  with  equal 
success,  but  the  event  must  be  observed  in  a  more  complicated  way 
and  over  a  longer  period  of  time.  The  first  ARPANET  objective 
was  "to  develop  techniques  and  obtain  experience  on 
interconnecting  computers  in  such  a  way  that  a  very  broad  class 
of  interactions  are  possible".  Not  just  techniques,  but  an 
entire  technology  of  packet  switching  has  been  developed,  and  an 
enormous  body  of  experience  has  been  developed  on  interconnecting 
computers  to  allow  a  broad  class  of  interactions. 

A  second  objective,  which  has  also  been  attained,  was  to 
improve  computer  research  productivity  through  the  development  of 
computer  resource  sharing. 

Another  objective  was  to  permit  the  linking  of  specialized 
computers  to  the  many  general  purpose  computer  centers.  Several 
major  specialized  computers  have  been  linked  over  the  ARPANET  to 
the  main  general  purpose  computer  centers  in  an  extremely 
successful  fashion. 
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At  a  technological  level,  the  overall  objectives  were  to 
construct  a  subnetwork  whose  reliability,  delay  characteristics, 
capacity,  and  cost  would  facilitate  resource  sharing  among  the 
computers  in  the  network;  and  to  understand,  design  and  implement 
the  host  protocols  to  permit  such  resource  sharing.  Such  a 
subnetwork,  consisting  of  over  50  nodes  and  stretching  from 
Hawaii  to  Norway,  was  successfully  constructed;  and  the  necessary 
host  protocols  to  allow  resource  sharing  among  the  connected 
computers  were  successfully  understood,  designed  and  implemented. 

The  demonstration  of  initial  net  operation  with  four  nodes 
and  later  the  extension  to  19  nodes  took  place  on  a  time  scale 
that  was  extremely  close  to  the  incremental  time  anticipated  in 
the  program  that  was  being  planned.  However,  as  the  success  of 
the  ARPANET  became  obvious  after  about  two  years,  decisions  were 
made  to  take  advantage  of  this  success  and  to  grow  the  net  to  a 
size  that  had  never  been  anticipated  at  the  outset;  growth  to 
more  than  50  nodes  over  a  geographic  area  from  Hawaii  *o  Norway 
was  certainly  not  originally  anticipated.  In  similar  fashion, 
the  costs  in  the  first  two  fiscal  years  of  the  program  were  very 
close  to  anticipated  costs,  but  as  decisions  were  made  to  expand 
the  scale  of  the  network,  the  cost  no  longer  followed  the 
original  program  plan. 
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In  short*  the  results  of  the  effort  were  eminently 
successful  and  far  more  than  adequately  met  the  objectives 
initially  stated.  The  success  of  the  program  far  exceeded  even 
the  most  optimistic  views  at  the  time  of  inception. 

3.2  Technical  Aspects  of  the  Effort  Which  Were  Successful  and 
Aspects  of  the  Effort  Which  Did  Not  Materialize  as  Originally 
Envisaged 

A  representative  list  of  the  aspects  of  the  ARPANET  programs 
which  were  technical  successes  follows: 

o  Powerful  computer-based  techniques  for  topological 

optimization  were  developed  and  the  choice  of  ARPANET 
topology  was  made  with  the  aid  of  these  tools. 

o  The  carriers  successfully  provided  high  reliability 

50K/sec  circuits. 

o  A  subnetwork  including  nodal  switching  IMPs  and  TIPs  was 
constructed  whose  performance,  reliability,  and  cost  did 
facilitate  resource  sharing. 

o  The  ARPANET  provides  a  convincing  demonstration  that 
adaptive  routing  algorithms  can  be  made  to  perform 
reliably  (e.g.,  in  a  globally  correct  manner  in  the  face 
of  local  failures),  efficiently  (e.g.,  adapting  to 
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changes  in  the  network  quickly  and  accurately),  and 
flexibly  (e.,?.f  accommodating  a  variety  of  circuit 
bandwidths  and  internode  distances)  without  excessive 
complexity  and  overhead. 

o  The  ARPANET  has  demonstrated  that  it  is  possible  to 

build  a  large  operational  network  in  such  a  way  that  the 
effects  of  component  failures  are  localized  rather  than 
"crashing"  or  otherwise  making  non-operational  large 
portions  or  all  of  the  network.  A  node  or  a  host  can 
fail  in  the  ARPANET  and  network  use  will  be  prevented 
for  only  the  few  users  directly  connected  to  that  node 
or  using  that  host. 

o  The  ARPANET  has  confirmed  the  theoretical  result  that 

networks  which  store-and-forward  packets  can  achieve 
delays  which  are  low  when  compared  to  the  delays 
incurred  in  the  computers  (hosts)  which  are  using  the 
network. 

o  The  ARPANET  has  demonstrated  that  a  network  can  be 
constructed  so  that  nodes,  lines,  traffic,  and  so  on  can 
be  added  or  deleted  without  major  upheavals  with  each 
addition. 
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o  The  ARPANET  has  demonstrated  that  it  is  possible  for  a 
network  to  control  and  operate  itself  for  hours  at  a 
time  without  explicit  control  from  a  control  center. 

o  The  ARPANET  has  demonstrated  new  techniques  for 
monitoring,  maintenance,  and  debugging.  The  nodes  in 
the  ARPANET  typically  operate  at  sites  where  there  is  no 
knowledgeable  person  available  locally.  The  nodes 

automatically  report  their  status  (via  the  network 
itself)  to  a  network  monitoring  center,  ar.d  maintenance 
and  debugging  (of  both  the  software  and  hardware)  are 
typically  carried  out  (again  via  the  network)  from  the 
monitoring  center. 

o  Possibly  the  most  difficult  task  undertaken  in  the 

development  of  the  ARPANET  was  the  attempt  --  which 
proved  successful  —  to  make  a  number  of  independent 
host  computer  systems  of  varying  manufacture,  and 
varying  operating  systems  within  a  single  manufactured 
type,  communicate  with  each  other  despite  their  diverse 
characteristics . 

o  A  set  of  host  protocols  was  hammered  out  between  the 

host  organizations  and  resulted  in  a  layered  structure 
of  host  protocols  that  did  facilitate  resource  sharing. 
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o  The  ARPANET  was  successfully  transferred  to  the  Defense 
Communications  Agency. 

Inevitably  there  were  technical  areas  which  proved  less 
tractable  than  had  been  hoped  and  some  kinds  of  network 
utilization  developed  more  slowly  than  had  been  anticipated. 

Problems  of  routing,  flow  control,  and  congestion  control  in 
the  subnetwork  turned  out  to  be  as  difficult  as 
theoreticians  had  anticipated,  and  the  algorithms  had  to  be 
modified  more  than  the  implementors  had  anticipated. 
Luckily,  the  improvements  in  the  algorithms  managed  to  stay 
slightly  ahead  of  the  growth  in  network  size  and  traffic 
and,  therefore,  difficulties  with  the  algorithms  never 
represented  an  impediment  to  resource  sharing  on  the 
network. 

It  proved  more  difficult  to  agree  on  the  specification  of 
adequate  host  protocols  than  had  been  originally  hoped. 
However,  the  eventual  design  of  the  host  protocols  was 
eminently  successful  and  provided  a  strong  base  for  resource 
sharing. 

The  development  of  a  widely-useful  on-line  Network 
Information  Center  (NIC)  proved  to  be  a  difficult  project. 
Although  very  interesting  and  important  computer-based  tools 
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for  information  handling  were  developed,  the  actual  timely 
collection  and  on-line  publication  of  detailed  descriptions 
of  resources  at  the  various  host  sites  proved  nearly 
intractable.  Further,  the  computer  tools  that  were 
developed,  while  rather  elegant,  were  not  so  clearly  cost 
effective  for  wide  scale  remote  use.  Eventually,  the  goals 
of  the  NIC  were  substantially  reduced  and  at  the  time  of  the 
ARPANET  transfer  to  DCA,  it  was  generally  necessary  to 
obtain  detailed  information  about  the  resources  at  a 
particular  site  directly  from  the  site. 

Many  different  kinds  of  resource  sharing  use  were 
anticipated  at  the  inception  of  the  ARPANET  program.  Of 
these,  the  remote  use  of  one  program  by  another  was  one  of 
the  more  distant  goals.  While  some  of  this  kind  of  resource 
sharing  has,  indeed,  taken  place,  such  use  has  grown  more 
slowly  than  other  network  uses. 


11-27 


faaaa no*  a  .  mi . 


mam&mSimmk 


Report  No.  4799 


Bolt  Beranek  and  Newman  Inc 


4.  APPLICATIONS  AND  CONSIDERATIONS  FOR  THE  FUTURE 

4.1  Conclusions  of  Technical  Feasibility 

The  ARPANET  project  proved  the  technical  feasibility  of 
achieving  reliable  high  performance,  cost  effective  digital 
communications  by  means  of  packet  switching  technology  and,  in 
turn,  the  technical  feasibility  of  operating  a  resource  sharing 
computer  network  based  on  this  technology.  The  ARPANET  project 
also  proved  the  feasibility  of  achieving  closely  knit  communities 
of  technical  interest  over  a  widespread  geographic  area;  it  is 
possible  that  this  social  feasibility  demonstration  is  as 
important  as  the  many  technical  feasibility  demonstrations. 

4.2  Recommendations  on  Additional  RAD  Requirements  and 
Opportunities 

It  is  clear  that  packet  switching  networks  have  a  very 
direct  application  to  command  and  control  and  a  significant 
research  opportunity  exists  in  attempting  to  investigate  such 
command  and  control  applications.  DARPA  is  already  proceeding  to 
develop  testbeds  whereby  packet  switching  technologies  and  other 
related  technologies  can  be  experimentally  employed  in 
cooperation  with  one  or  more  of  the  military  services.  There  is 
a  clear  requirement  for  improved  command  and  control  and  the 
packet  switching  technology  developed  in  the  ARPANET  provides  a 
major  opportunity  to  make  progress  towards  this  goal. 
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At  a  deep  technical  level,  the  proper  design  of  host 
operating  systems  for  efficient  and  cost  effective  connection  to 
networks  is  still  in  the  area  of  significant  research  and 
development  opportunity.  In  the  ARPANET  project,  the  network 
connections  were  "add-ons”  and  it  is  clearly  time  to  mount  the 
more  detailed  investigation  of  how  best  to  accomplish  such 
connections. 

The  techniques  for  remote  control  of  computers  in  the  field 
developed  within  the  ARPANET  project  are  probably  more  broadly 
applicable  to  the  management  of  computer  resources  in  other  areas 
of  the  Defense  Department.  These  remote  management  techniques 
represent  another  opportunity  which  has  grown  out  of  the  ARPANET 
experience. 

It  was  earlier  indicated  that  it  was  not  practical  for  the 
Network  Information  Center  in  the  ARPANET  to  keep  current  on-line 
detailed  descriptions  of  program  resources  available  within  the 
host  community.  This  difficulty  in  turn  represents  a  research 
and  development  opportunity  for  the  future.  Specifically, 
research  and  development  is  required  on  how  to  properly  describe 
computer  programs  and  how  to  create  standards  for  such 
descriptions  such  that  it  will  be  easier  to  create  compendia  of 
available  resources.  In  other  words,  despite  the  great  success 
of  the  ARPANET,  the  basic  resource  sharing  problem  still 
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represents  a  fertile  area  for  research  and  development.  Now  that 
the  networking  technology  itself  is  '•given” ,  attempts  at 
description  and  documentation  of  host  resources  might  more  easily 
yield  to  a  research  and  development  effort. 
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5.  PROGRAM  IMPACT  AND  ASSESSMENT  OF  TECHNOLOGY  DEVELOPED 

5.1  Service  Use  of  Technology 

The  ARPANET  technology  is  being  successfully  transferred  to 

the  rest  of  DoD.  Not  only  was  it  possible  to  transfer  the 

ARPANET  itself  to  the  Defense  Communications  Agency  (Code  535),* 
but  the  Defense  Communications  Agency  has  already  embarked  on  the 
procurement  of  its  primary  backbone  major  communications  system 
of  the  future  —  AUTODIN  II  —  based  very  specifically  on  the 
packet  switching  technology  developed  in  the  ARPANET.  Even 
beyond  this  very  major  step  all  three  services  are  actively 

involved  in  investigations  of  packet  switching  technology  for 

their  specific  needs. 

5.2  Impact  on  Non-DoD  Programs 

In  just  the  very  short  time  since  the  inception  of  the 
ARPANET  this  technology  has  already  resulted  in  the  formation  of 
a  new  industry:  a  private  sector  development  of  "value  added" 
packet  switching  networks.  Two  corporations,  Telenet  and 
Tymshare,  are  currently  marketing  packet  switched  communications 
to  the  general  public  as  common  carriers  under  the  Communications 
act  of  1934.  This  program  therefore  has  also  had  direct  and 
immediate  exploitation  in  the  commercial  sector.  In  addition, 

*  Now  DCA  Code  252  (December  1981). 
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all  over  the  worlc  new  communications  systems  are  being  designed 
and  built  to  take  advantage  of  the  packet  switching  technology 
demonstrated  by  the  ARPANET  project.  At  least  three  countries  — 
Great  Britain,  France  and  Canada  —  have  major  national  PTT- 
sponsored  packet  switching  networks  either  already  operating  or 
under  development  and  many  other  countries  are  actively  pursuing 
this  technology. 

5.3  Applications  of  the  Program  Results 

The  most  specific  result  of  the  ARPANET  Program  has,  of 
course,  been  the  ARPANET  itself;  and  the  ARPANET  itself  is 
currently  fully  operational  under  the  management  of  the  Defense 
Communications  Agency  and  is  actively  serving  thousands  of 
individuals  on  a  daily  basis. 

5.4  Advance  in  the  Sta te-of-the-Art 

The  ARPANET  program  has  represented  a  first-rank  advance  in 
the  state-of-the-art  of  communications  and  the  state-of-the-art 
of  computer  technology.  The  greatest  advance  has  been  in  the 
provision  of  cost  effective,  reliable,  high  performance  digital 
communications,  but  very  significant  state-of-the-art  advances 
have  also  taken  place  in  many  other  areas,  such  as  topological 
optimization,  routing,  multiprocessor  technology,  protocols  for 
resource  sharing  between  programs,  network  mail  systems,  and 
remote  computer  management. 
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6.  BIBLIOGRAPHY  OF  REPORTS 

There  has  been  an  enormous  amount  of  literature  about  the 
ARPANET  in  particular,  and  about  resource  sharing  computer 
networks  in  general.  Literally  an  industry  has  been  formed  in 
the  space  of  two-thirds  of  a  decade  and  the  amount  of  literature 
reflects  this  really  unusual  metabolism. 

The  initial  seminal  papers  on  the  ARPANET  were  presented  in. 
May  of  1970  at  the  AFIPS  Spring  Joint  Computer  Conference  in 
Atlantic  City  and  are  published  in  the  Proceedings  of  that 
conference  (AFIPS  Conference  Proceedings,  Vol  36,  AFIPS  Press, 
210  Summit  Avenue,  Montvale,  New  Jersey  07645).  Another  early 
ARPANET  session  was  held  at  the  1972  Spring  Joint  Computer 
Conference  in  Atlantic  City  and  these  papers  are  published  in 
AFIPS  Conference  Proceedings,  Vol.  40.  These  two  early  sets  of 
papers  represent  a  sensible  introduction  to  the  early  notions 
about  and  plans  for  the  ARPANET. 

A  sizeable  bibliography  and  index  to  publications  about  the 
ARPANET  was  published  in  1976  with  DARPA  support:  "Selected 
Bibliography  and  Index  to  Publications  About  the  ARPANET",  Becker 
and  Hayes  Inc.,  February  1976,  185  pa^s,  AD-A026900,  This 
document  represents  the  most  complete  available  listing  about 
ARPANET  publications,  but  it  i3  a  bibliography  only  and  does  not 
provide  help  in  trying  to  select  which  of  the  many  references 
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would  be  suitable  to  look  at  in  what  order.  Another  bibliography 
on  the  literature  of  resource  sharing  computer  networks  m 
general  (not  just  the  ARPANET)  has  been  issued  by  the  U.S. 
Department  of  Commerce,  National  Bureau  of  Standards:  ’’Annotated 
Bibliography  of  the  Literature  on  Resource  Sharing  Computer 
Networks”,  NBS  Special  Publication  384,  revised  1976.  This 
document  is  also  not  very  useful  in  helping  the  reader  decide 
what  is  important  and  what  is  not. 

Several  volumes  of  reprints  have  been  published  which  deal 
with  computer  networks  in  general  (rather  than  the  ARPANET 
specifically),  but  which  attempt  to  collect  together  the 
important  papers  themselves  and  provide  a  much  easier  entry  to 
the  literature  than  the  large  bibliographies,:  (1)  "Advances  in 
Computer  Communications",  Wesley  W.  Chu,  Artech  House,  Inc.,  610 
Washington  St. ,  Dedham,  Massachusetts  02026,  1974;  (2)  "Computer 
Communications",  edited  by  Paul  E.  Green,  Jr.  and  Robert  W. 
Lucky,  IEEE  Press,  345  East  47th  Street,  New  York,  New  York 
10017,  1975;  and  (3)  "Computer  Networking",  edited  by  Robert  P. 
Blanc  and  Ira  W.  Cotton,  IEEE  Press,  345  East  47th  Street,  New 
York,  New  York  10C17,  1976. 

A  very  hefty,  but  fairly  readable  compendium  with  a  very 
large  list  of  references  is  "Infotech  State  of  the  Art  Report  24, 
Network  Systems  and  Software",  Infotech  International  Ltd., 
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Nicholson  House,  Maidenhead,  Berkshire,  England,  1975.  This 
document  deals  with  far  more  than  just  the  ARPANET,  but  it  tries 
to  put  the  ARPANET  in  context  with  other  current  work  as  of  1975. 

There  are  three  important  reference  documents  which  have 
been  prepared  for  the  Defense  Communications  Agency  by  the 
Network  Information  Center  at  Stanford  Research  Institute,  Menlo 
Park,  California  94025  which  are  specifically  addressed  to 
various  aspects  of  the  ARPANET:  (1)  "ARPANET  Resource  Handbook", 
NIC  39335,  December  1976;  (2)  "ARPANET  Protocol  Handbook",  NIC 
7104,  Revision  1,  April  1976;  (3)"ARPANET  Directory",  NIC  36437, 

July  1976. 

A  textbook  on  computer  networks  is  Davies  and  Barber, 
"Communication  Networks  for  Computers",  John  Wiley,  1973.  A 
special  issue  of  the  IEEE  proceedings  on  Packet  Communications 

was  issued  in  November,  1978. 
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CHAPTER  III: 


A  HISTORY  OF  THE  ARPANET  PROJECT 
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1 .  HISTORY 


The  DARPA  Computer  Network,  or  ARPANET  as  it  has  come  to  be 
called,  consists  of  IMPs,  lines,  and  hosts  as  shown  in  Figure  1. 
The  IMPs  (short  for  Interface  Message  Processors)  are  small, 
special-purpose  computers  connected  to  each  other  by  telephone 
lines.  The  hosts  are  a  heterogeneous  collection  of  computers 


Figure  1  —  IMPs,  Lines,  and  Hosts 

used  for  a  variety  of  applications.  The  IMPs  provide  a 
communications  subnetwork  through  which  hosts  communicate  with 
each  other,  much  as  the  commercial  telephone  system  provides  a 


III-2 


Report  No.  4799 


Bolt  Beranek  and  Newman  Inc. 


communications  subnetwork  through  which  humans  communicate.  In 
other  words,  the  IMPs  and  lines  make  up  a  communications  utility 
of  which  the  hosts  are  users.  Each  host  is  connected  to  one  IMP; 
one  IMP  may  have  several  hosts  connected  to  it. 

When  data  is  to  be  sent  from  one  host  to  another,  the  data 
is  broken  into  discrete  entities  called  '’messages”  at  the  sending 
host,  which  is  also  called  the  "source  host".  Each  message 
consists  of  some  data  and  an  address.  The  address  specifies  to 
which  host  the  data  is  to  be  sent,  that  is,  the  "destination 
host".  Successive  messages  are  passed  from  the  source  host  to 
its  IMP,  where  they  are  broken  into  smaller  entities  called 
"packets",  each  of  which  carries  the  same  address  as  the  message. 
The  packets  are  routed  from  IMP  to  IMP  across  the  network  until 
they  arrive  at  the  IMP  to  which  the  destination  host  is 
connected,  where  the  original  messages  are  reconstructed  from  the 
packets,  and  passed  to  the  destination  host. 

The  ARPANET  was  conceived  in  the  middle  to  late  1960s  as  a 
project  to  be  sponsored  by  the  Information  Processing  Techniques 
Office  of  the  Defense  Advanced  Research  Projects  Agency  (DARPA) 
of  the  U.S.  Department  of  Defense  (DoD).  Construction  of  the 
network  began  ip  1969.  By  1975  the  network  had  gone  through 
several  stages  of  development  and  for  all  intents  and  purposes 
had  become  an  operational  Department  of  Defense  computer  network. 
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In  1975,  control  of  the  network  was  transferred  from  DARPA  to  the 
U.S.  Defense  Communications  Agency,  an  agency  better  suited  to 
the  administration  of  a  working  facility. 

The  ARPANET  is  a  major  development  in  the  evolution  of 
computer  communications.  Our  purpose  here  is  to  present  the 
history  of  the  ARPANET:  why  it  was  built,  who  helped  build  it, 
the  major  design  decisions  (and  some  of  the  minor  ones) 
associated  with  it,  its  evolutionary  development,  its  maturity, 
and  why  it  is  important. 
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1 . 1  Background 

1.1.1  The  RAND  Study  of  Distributed  Communications  Networks 

One  of  the  most  important  early  studies  of  computer  networks 
was  performed  by  Paul  Baran  and  his  colleagues  at  the  RAND 
Corporation  in  the  early  1960s.  Many  concepts  central  to  the 
later  development  of  the  ARPANET  and  other  computer  networks  were 
first  described  in  the  series  of  reports  published  by  RAND  in 
1964  (a  list  of  these  reports  is  given  in  the  bibliography  at  the 
end  of  this  subsection).  These  ideas  include  the  improved 
reliability  of  a  distributed  network  structure  over  a  centralized 
or  star  network  and  over  so-called  decentralized  networks  made  up 
of  a  collection  of  smaller  star  networks.  Extensive  studies  were 
undertaken,  including  simulation  of  some  grid  networks,  to 
determine  how  "survivable"  a  distributed  network  could  be 
expected  to  be  after  heavy  node  and  link  failures.  This  study 
was  particularly  concerned  with  the  question  of  keeping  a  high 
percentage  of  the  network  available  and  performing  well  in  the 
face  of  enemy  attacks  on  the  network,  from  the  point  of  view  of 
its  suitability  for  Department  of  Defense  applications. 

In  specifying  closely  the  engineering  details  of  what  was 
called  the  "Distributed  Adaptive  Message  Block  Network",  Baran 
anticipated  many  of  the  developments  in  practical  networks  that 
came  a  full  decade  later.  In  ♦‘he  Distributed  Adaptive  Message 
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Elock  Network,  a  "multiplexing  station"  connects  up  to  1024 
terminals  of  widely  differing  characteristics.  Automatic 
user-to-user  cryptography  is  integrated  into  the  network 
switching  technique  to  ensure  efficiency.  Both  satellite  links 
and  low-cost  microwave  relay  systems  are  suggested  as  techniques 
for  providing  the  n<»tvork  with  very  high  data  rate  circuits.  The 
concept  of  a.  "message  block"  Is  introduced:  a  packet  of  up  to 
1024  bits  of  header  and  data,  which  is  the  unit  of  data 
transferred  in  the  network.  One  of  the  most  interesting  aspects 
of  this  study  is  that  it  concluded  that  a  large-scale  digital 
transmission  network  was  not  only  feasible  but  also  highly 
cost-effective,  and  proposed  that  many  of  the  switching  functions 
be  implemented  in  hardware.  Baran  was  considering  ways  of  making 
extremely  reliable  networks,  and  so  preferred  simple  solutions 
and  reliable  hardware  where  possible. 

The  following  bibliography  includes  the  entire  set  of  eleven 
reports  in  the  original  RAND  study  as  well  as  two  published 
papers  resulting  from  that  set  and  from  two  later  reports.  An 
annotated  version  of  this  bibliography  is  included  in  "Adaptive 
Routing  Algorithms  for  Distributed  Computer  Networks"  (John  M. 
McQuillan,  BBN  Report  No.  2831,  May  1974,  pp.  14-17). 
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1.1.2  The  Lincoln/SDC  Experiment 

The  notion  of  linking  computers  into  a  network  was 
envisioned  by  J.C.R.  Licklider  long  before  the  ARPANET  become  a 
reality.  Licklider  was  the  first  director  of  DARPA  IPTO. 

In  1965  Thomas  Marill  and  his  colleagues  at  Computer 
Corporation  of  America  (in  Cambridge,  Massachusetts)  were 
commissioned  by  M.I.T.  Lincoln  Laboratory  to  study  the  concept  of 
computer  networking.  The  primary  technical  contact  at  Lincoln 
Laboratory  was  Lawrence  Roberts,  who  played  an  active  role  in  the 
network  study;  and  the  contract  was,  in  fact,  a  subcontract  under 
the  Laboratory's  DARPA  contract.  The  study  was  done  in  late  1965 
and  a  report  on  the  study  was  issued  in  1966  ("A  Cooperative 
Network  of  Time-Sharing  Computers",  Thomas  Marill,  Computer 
Corporation  of  America,  Technical  Report  No.  11,  June  1,  1966;  a 
later  paper  of  the  same  name  authored  by  Thomas  Marill  and 
Lawrence  Roberts  also  appeared  in  the  Proceedings  of  the  AFIP5 
1966  Spring  Joint  Computer  Conference,  pp.  425-431).  The  report 
examined  the  basic  idea  of  computer  networking,  considered  the 
available  communications  techniques  and  software  problems,  and 
recommended  that  a  three-computer  experimental  network  be 
constructed.  The  report  suggested  linking  three  existing 
computers,  the  AN/FSQ-32  at  Systems  Development  Corporation,  the 
IBM  7094  at  MIT's  Project  MAC,  and  Lincoln  Laboratory's  TX-2. 
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Later  in  1966,  CCA  received  another  contract  to  carry  out  the 
linking  of  the  Q-32  and  the  TX-2.  The  Q-32  and  TX-2  were  in  fact 
linked  together,  and  the  link  was  demonstrated.  Later  a  small 
Digital  Equipment  Corporation  machine  at  DARPA  was  added  to  this 
network,  by  now  known  as  ”The  Experimental  Network”.  It  is 
noteworthy  that  The  Experimental  Network  linked  host  computers 
directly,  and  did  not  use  IMPs. 


*  • 
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1.1.3  The  NPL  Data  Network 

Another  early  major  network  development  which  affected 
development  of  the  ARPANET  was  undertaken  at  the  National 
Physical  Laboratory  in  Middlesex,  England,  under  the  leadership 
of  D.  W.  Davies.  The  broad  system  design  of  the  NPL  Data 
Network,  as  it  was  called,  was  first  published  in  1967,  and  bears 
a  resemblance  to  the  network  proposed  by  Paul  Baran  at  RAND,  and 
to  the  ARPANET.  The  NPL  Data  Network  was  specified  to  be  a 
packet-switching  network  and  was  to  have  a  hierarchical 
structure.  It  was  proposed  that  "local  networks"  be  constructed 
with  "interface  computers"  which  had  responsibility  for 
multiplexing  among  a  number  of  user  systems  and  for  communicating 
with  a  "high  level  network".  The  latter  would  be  constructed 
with  "switching  nodes"  connected  together  with  megabit  rate 
circuits. 

Considerable  detail  about  the  NPL  Data  Network  may  be  found 

in  the  textboGk,  written  by  two  of  the  network's  designers, 

-entitled  CgfflBmis fltlons _ Networks  for  Computers  (D.W.  Davies  and 

D.L.A.  Barber,  John  Wiley  &  Sons  publishers,  1973).  A 

« 

|  bibliography  of  published  papers  resulting  from  the  NPL  Data 

Network  effort  follows. 
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1.2  The  Events  of  1967  and  1968 

The  years  1967  and  1968  were  spent  promoting  interest  in  the 
ARPANET  project  both  within  the  government  and  with  IPTO 
contractors,  deciding  the  fundamental  structure  of  the  network, 
writing  a  request  for  quotation,  selecting  a  contractor,  and 
other  related  activities. 

In  early  1967,  work  began  on  the  conventions  to  be  used  for 
exchanging  messages  between  any  pair  of  computers  in  the  proposed 
network,  and  also  on  consideration  of  the  kinds  of  communications 
lines  and  data  sets  to  be  used.  In  particular,  it  was  decided 
that  the  inter-host  communication  "protocol"  would  include 
conventions  for  character  and  block  transmission,  error  checking 
and  retransmission,  and  computer  and  user  identification.  Frank 
Westervelt,  then  of  the  University  of  Michigan,  wrote  a  position 
paper  on  these  areas  of  communication,  an  ad  hoc  "Communication 
Group"  was  selected  from  among  the  institutions  represented,  and 
a  meeting  of  the  group  scheduled. 

The  plan  considered  throughout  much  of  the  Michigan  meeting 
was  to  connect  all  of  the  computers  by  dial-up  telephone  lines 
and  data  sets  so  as  to  allow  any  computer  to  establish 
communication  with  any  other  computer  using  a  line-switching 
technique  (i.e.,  calling  it  up  on  the  telephone).  A  small 
program  in  each  computer  would  interface  to  the  data  set  and 
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phone  line  and  when  given  a  message  to  -another  computer,  the 
small  interface  program  would  perform  a  "message-switching”  and 
transmission  function,  deciding  how  to  actually  reach  the  other 
computer  and  transmitting  the  message  to  it.  Wes  Clark,  then  of 
Washington  University,  is  credited  with  proposing  at  some  point 
in  the  meeting  that  a  small  computer  be  inserted  between  each 
participant's  computer  and  the  phone  line.  This  concept  was 
further  refined  within  DARPA,  and  the  concept  of  the  Interface 
Message  Processor  or  IMP  emerged. 

The  IMP  would  perform  the  functions  of  dial-up,  error 

checking,  retransmission,  routing,  and  verifications  on  behalf  of 

the  participants'  computers  (which  we  shall  hereafter  call 

hosts).  Thus  the  IMPs  plus  the  telephone  lines  and  data  sets 

would  constitute  a  "message-switching  network".  The  protocols 

which  were  to  be  established  would  define  the  communications 

formats  between  the  IMPs.  The  interface  between  a  host  and  an 

IMP  would  be  a  digital  interface  of  a  much  simpler  sort  requiring 

no  host  consideration  of  error  checking,  retransmission,  and 

routing.  It  was  clearly  noted  that  the  major  disadvantage  of 

inserting  the  IMP  was  the  cost  of  installation  of  another 

computer  beside  each  host.  The  major  advantage  of  inserting  the 

IMPs  was  recognized  to  be  the  fact  that  a  unified, 

straightforward  network  design  could  be  made  and  implemented 

* 

without  undue  consideration  of  the  variations  and  constraints  of 
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the  host  computers.  Further,  as  the  network  evolved,  it  would  be 
much  simpler  to  modify  the  network  of  IMPs  than  to  modify  all  the 
host  computers.  Finally,  the  IMPs  would  relieve  the  hosts  of  the 
communications  burdens  they  were  initially  scheduled  to  carry. 
It  was  also  noticed  that  if  necessary  IMPs  could  be  located  at 
strategic  connection  points  within  the  network  to  concentrate 
messages  over  cross-country  phone  lines,  a  network  of  IMPs  was 
likely  to  be  implemented  faster  than  a  network  of  directly 
connected  hosts,  and  the  network  of  IMPs  provided  a  distinct 
network  entity  which  would  be  useful  in  presenting  the  network 
publicly. 

By  October  1967  the  proposed  network  was  becoming  known  as 
the  ARPA  Computer  Network,  or  ARPANET  for  short.  A  variety  of 
topics  were  under  discussion  at  that  time  including  message 
formatting,  message  protocols,  dynamic  routing  and  message 
propagation,  queuing,  error  control,  measurements,  and 
IMP-to-host  communication.  It  was  decided  that  50  Kb 
communications  lines  would  be  used  because  of  the  vastly  improved 
response  time  which  could  be  obtained  with  these  lines  as  opposed 
to  the  previously  proposed  2.4  Kb  lines.  The  50  Kb  lines  were  to 
be  leased,  eliminating  the  slow  dial-up  procedure.  The  nature  of 
the  telephone  tariffs  available  to  the  government  made  use  of  50 
Kb.  lines  affordable.  With  each  IMP  normally  permanently 
connected  to  at  least  two  other  IMPs  via  the  leased  50  Kb  lines. 
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the  IMPs  were  to  use  store-and-forward  techniques  to  provide  fast 
message  handling.  Each  IMP  was  to  accept  messages  of  up  to  8,000 
bits  from  its  host  computer  and  to  break  this  into  1  ,000-bit 
packets.  Each  packet  was  to  be  treated  independently  and  routed 
on  one  of  the  two  or  more  inter- IMP  lines.  When  a  packet  arrived 
at  an  IMP,  it  was  to  be  stored,  error  checked,  and  routed  on  to  a 
further  In?.  At  its  destination,  packets  would  be  held  until  an 
entire  message  could  be  assembled  and  then  delivered  to  the 
destination  host.  With  an  average  of  say  three  lines  per  IMP,  it 
was  expected  that  approximately  three  store-and-forward  stages 
would  be  necessary  to  get  a  message  from  any  one  of  twenty 
locations  to  any  other. 

In  July  1968,  an  RFQ  for  the  network  was  mailed  out  to 
prospective  bidders.  Twelve  proposals  were  received  by  the  Agent 
(DSSW).  Four  bidders  were  rated  within  the  zone  of  contention  to 
receive  the  IMP  contract,  and  supplementary  technical  briefings 
were  requested  from  each  of  these  bidders.  Final  negotiations 
were  carried  out  with  two  finalists,  and  one  was  chosen  in  the 
week  before  Christmas,  1968.  The  contract  was  awarded  and  work 
began  the  second  day  of  the  New  Year  in  1969. 
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1 .3  Key  Aspects  of  the  RFQ 


It  was  specified  that  responses  to  the  RFQ  would  be 
evaluated  on  four  criteria  in  addition  to  cost: 

1.  Understanding  and  depth  of  analysis  of  technical 
problems  involved. 

2.  Availability  of  qualified,  experienced  personnel  for 
assignment  to  software,  hardware,  and  installation  of 
the  system. 

3.  Estimated  functional  performance  and  choice  of  hardware. 

4.  General  quality,  responsiveness,  and  corporate 
commitment  to  the  network  concept. 

The  RFQ  had  provision  for  a  bidders  conference  and  stated 
that  there  would  be  no  other  opportunity  for  bidders  to  discuss 
technical  issues  with  the  government.  Bidders  were  asked  to 
provide  a  system  design  for  a  nineteen-IMP  network,  but  to  price 
a  four-IMP  network.  A  thirteen-month  performance  period  was 
requested  to  include  design,  construction  of  a  prototype  IMP,  and 
implementation  and  installation  of  four  operational  IMPs.  The 
four  IMPs  were  to  be  installed  nine  months  after  start  of  the 
contract  with  the  contractor  supporting  them  in  the  field  for 
three  months  after  installation.  The  contractor  was  required  to 
take  full  system  responsibility,  although  subcontracting  a 
portion  of  the  work  wa3  a  possibility. 
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I.  Network  Description 

A.  Introduction 

B.  Functional  Description 

1 .  The  User  Subnet 

2.  The  Communication  Subnet 

C.  Functional  Description  of  the  IMPS 

1 .  Breaking  of  Messages  into  Packets 

2.  Management  of  Message  Buffers 

3.  Routing  of  Messages 

4.  Generation,  Analysis  and  Alteration  of 
Formatted  Messages 

5.  Coordination  of  Activities  with  Other  IMPS 

6.  Coordination  of  Activities  with  its  HOST(s) 

7.  Measurement  of  Network  Parameters  and 
Functions 

8.  Detection  and  Disposition  of  Faults 

9.  IMP  Software  Separation  Protection 

D.  The  HOST-IMP  Interfaces 

E.  The  IMP-CARRIER  Interfaces 

F.  Network  Performance  Characteristics 

1 .  Message  Delay 

2.  Reliability 

3.  Network  Capacity 

4.  Network  Model 

G.  HOST-HOST  Characteristics 

H.  IMP-Operator  Interface 

II.  Network  Contractor  Performance 

III.  Elements  of  System  Design 
Appendix 

A.  ARPA  Network  Nodes 

B.  ARPA  Network  Topology 

C.  IMP  Delivery  Schedule 

D.  Input  and  Output  Facilities  for  the  IMP  Operator 

E.  ARPA  Network  Data  Rates  Between  Nodes  in  Kilobits/sec. 

F.  Data  Communications  Conventions 

G.  Routing 


Figure  2:  Table  of  Contents  from  RFQ  Statement  of  Work 
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The  IMP  specification  clearly  delineated  the  division  of 
responsibilities  among  host  sites,  IMP  contractor,  and  telephone 
company.  Each  individual  host  site  was  responsible  for  designing 


and  implementing 

for 

its 

own 

convenience 

the 

hardware 

and 

software  necessary 

to 

attach 

the 

host  to  the 

network,  and 

the 

hardware  and  software 

to  utilize 

other  hosts  on 

the 

network. 

The 

telephone  company  was  to  be  responsible  for  providing  necessary 
circuits,  data  sets,  and  line  conditioning  equipment  utilized  by 
the  network.  The  IMP  contractor  was  to  be  responsible  for 
providing  necessary  hardware  and  software  to  connect  IMPs  to  each 
other  using  the  circuits  supplied  by  the  telephone  company  and  to 
connect  IMPs  to  hosts,  as  well  as  providing  hardware  and  software 
necessary  to  implement  the  procedures  which  allowed  creation  of  a 
network  of  IMPs  capable  of  forwarding  messages  from  one  host  to 
another. 

The  functional  description  of  an  IMP  specified  the  use  of 
messages  not  longer  than  8192  bits  which  would  be  broken  into 
packets  of  not  more  than  1024  bits.  Messages  were  limited  in 
size  to  make  them  manageable  for  the  hosts.  Shorter  packets  were 
used  to  reduce  the  probability  of  transmission  error  with  the 
attendant  necessity  for  retransmission.  It  was  noted  that  IMP 
provision  of  message  and  packet  buffer  space  would  permit  speed 
conversions  to  take  place,  provide  queuing  space  in  the  face  of 
delays,  and  permit  retransmission  in  the  event  of  erroneous 
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transmission.  A  routing  algorithm  was  hypothesized  which  would 
take  into  account  the  connectivity  of  the  network,  IMP  and  line 
busyness,  and  message  priority,  and  use  this  information  to 
forward  a  packet  to  the  next  IMP  on  a  path  to  the  ultimate 
destination;  periodic  updates  baaed  on  exchange  of  routing  and 
loading  information  with  other  IMPs  and  hosts  was  also 
hypothesized.  An  IMP  was  to  coordinate  its  activities  with  other 
IMPs  and  its  hosts  and  perhaps  other  special  hosts.  IMPs  were  to 
take  messages  from  a  local  host  at  the  IMP’S  convenience,  but  to 
send  messages  to  a  local  host  at  the  host's  convenience.  The 
IMPs  were  to  be  able  at  selected  times  to  measure  selected 
network  parameters  and  to  trace  the  movements  of  selected 
messages  through  the  network.  The  data  resulting  from  these 
measurements  and  tracings  was  to  be  capable  of  transmission  to  a 
host,  and  the  measurement  activity  was  to  be  capable  of 
initiation  and  termination  by  a  host  or  another  IMP.  The  IMPs 
were  to  detect  and  recover  from  various  IMP,  host,  and  line 
failures.  In  particular,  it  was  to  be  possible  to  stop,  start, 
examine,  or  reload  IMPs  from  selected  network  hosts.  Finally,  it 
was  thought  that  at  each  IMP  site,  it  would  be  possible  for 
special  host-specific  code  to  be  provided  by  host  site 
programmers,  and  thus  it  was  desirable  to  protect  the  rest  of  the 
IMP  from  the  portion  that  the  host  personnel  could  access  and 
program. 
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There  were  two  particularly  interesting  aspects  of  the 
host-to-IMP  interface:  1)  a  standard  host  interface  was  to  be 
specified  rather  than  a  different  one  for  each  host;  2)  each 
bidder  was  to  consider  the  cost  of  providing  interfaces  to 
multiple  hosts  per  IMP,  although  only  one  host  interface  was 
required;  and  3)  sufficient  program  space  was  to  be  left  to  do 
host-specific  character  code  conversion  and  repacking  of  binary 
messages. 

The  interface  between  an  IMP  and  its  telephone  lines  was 
required  to  have  hardware  to  sense  characters,  detect  control 
characters,  calculate  and  check  the  24-bit  CRC,  provide  a 
real-time  clock  with  20  microseconds  resolution,  and  provide 
fault  and  status  information.  Further,  the  IMP  was  to  be 
optimized  to  handle  three  lines  but  be  capable  of  handling  six. 

Several  network  performance  characteristics  were  specified. 
The  average  message  delay  for  a  short  message  to  go  from  a  source 
IMP  to  a  destination  IMP  was  to  be  less  than  one-half  second  for 
a  fully  loaded  network.  The  probability  of  lost  messages  and 
message  errors  was  to  be  very  low.  Interestingly,  network 
capacity  was  considered  third  in  order  of  importance  and  was 
defined  to  be  the  maximum  bit  rate  that  can  be  input  at  every 
node  and  still  have  the  message  delay  remain  less  than  one-half 
second;  a  20  Kb  network  capacity  was  hoped  for.  A  network  model 
wa3  presented. 
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Host-to-host  traffic  flows  were  estimated  and  it  was 
hypothesized  that  there  would  be  a  trimodal  distribution  of 
traffic  type  (high  rate  and  short  length,  medium  rate  and  medium 
length,  and  low  rate  and  long  length). 

In  addition  to  considering  the  option  of  multiple  hosts 
connected  to  an  IMP,  the  bidder  was  also  asked  to  consider  the 
provision  of  memory  protection  to  facilitate  simultaneous  IMP 
operation  and  checkout  of  new  software,  and  to  consider  what 
additional  hardware  and  software  would  be  necessary  for  an  IMP  to 
provide  a  terminal  concentration  capability  for  its  host  or  for 
the  network  (i.e.,  no  host,  just  terminals). 
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1.4  Chronological  Development,  1969  to  1975 

Within  a  year  after  the  award  of  the  IMP  contract,  the  first 
IMPs  were  installed  in  the  field.  Hosts  were  connected  to  these 
first  IMPs  and  a  series  of  network  measurements  was  undertaken. 
Host  software  was  written,  hosts  began  to  communicate  with  each 
other,  more  IMPs  and  hosts  were  added,  and  gradually  the  ARPANET 
became  an  operating  entity.  After  six  years  of  development  and 
operation  the  network  was  no  longer  best  suited  to  management  by 
an  agency  with  the  charter  to  sponsor  advanced  research  and 
development,  and  thus  network  management  was  transferred  to  the 
Defense  Communications  Agency.  In  the  following  subsections  we 
describe  the  happenings  of  the  years  1969  to  1975. 
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1.4.1  The  Groups  and  the  Key  People 

The  ARPANET  development  was  a  joint  effort  of  many 
individuals  and  institutions,  all  responsive  to  DARPA's 
direction.  Before  we  describe  the  ARPANET  development  further, 
it  is  best  to  list  the  principal  "players"  and  briefly  describe 
their  areas  of  responsibility. 
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1.4. 1.1  Management  and  Administration  of  the  Network: 

DARPA  IPTO,  DSS-W,  RML,  and  DECCO 

Naturally,  DARPA  IPTO  played  a  major  role  in  the  development 
of  the  network.  The  director  of  IPTO  at  the  time  was  Robert 
Taylor,  who  provided  the  necessary  support  as  well  as 
encouragement  in  helping  to  fund  the  network.  Lawrence  Roberts 
was  the  initial  program  manager  for  the  network  program.  Roberts 
promoted  the  network,  made  certain  key  architectural  decisions, 
led  the  evaluation  team  which  selected  the  IMP  contractor  and 
selected  other  involved  contractors.  He  also  succeeded  Taylor  as 
director  of  IPTO. 

While  IPTO  set  policy  for  the  network,  made  decisions  about 
who  would  join  the  network,  etc.,  IPTO  did  not  run  the  network 
day  by  day.  BBN  provided  day-by-day  operation  and  maintenance  of 
the  network,  and  BBN  and  the  other  contractors  involved  carried 
out  much  of  the  day-by-day  business  of  the  network  among 
themselves  without  need  for  daily  IPTO  supervision. 

Initially  IPTO  itself  took  care  of  ordering  telephone  lines, 
filling  out  forms  which  were  sent  to  DECCO  for  procurement  from 
and  execution  by  the  various  telephone  companies.  Also,  IPTO 
initially  did  its  own  technical  monitoring  of  the  various 
contractors  associated  with  the  ARPANET;  DSS-W  was  used  as  the 
procurement  agency  for  the  contractors.  Eventually,  attempting 
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torid  Itself  of  the  routine,  tedious  aspects  of  ordering 
communications  lines  and  providing  technical  monitoring  of 
contractors,  IPTO  shifted  the  DSS-M  functions,  routine  dealings 
with  DECCO,  and  some  contractor  technical  monitoring 
Measurements  Laboratory  at  Patrick  Air  Force  Ease  In  Florida;  in 
addition  to  having  a  procurement  capability,  RHL  also  had 
technical  support  capability. 

There  were  several  other  members  of  the  IPTO  staff  who  have 

been  prominent  in  the  management  of  the  ARPANET  besides  Roberts. 

ni  n  D%  Carlstrom,  S* 

These  have  included  A.  Blue, 

_  c  Walker,  and  D.  Russell, 

Crocker,  B.  Dolan,  C.  Fields,  S.  Walker, 

.  w  TPTn  nf ten  olayed  a  strong  technical 

As  mentioned  above,  IPTO  often  pi  y 

_  tptd  sta^f  wrote  a  number 

role  in  the  ARPANET,  and  members  of  the  IPT 

..  hdp&hft  activity.  Several  of  these 
of  papers  describing  the  ARPANET  activity 

papers  are  listed  in  the  following  bibliography. 


III-27 


Report  No.  4799 


Bolt  Beranek  and  Newman  Inc 


Bibliography 

T.  Marill  and  L.  G.  Roberts,  "Toward  a  Cooperative  Network  of 
Time-Shared  Computers,"  Proceedings  FJCC  1966,  p. 425-431. 

L.  G.  Roberts,  "Multiple  Computer  Networks  and  Intercomputer 
Communication,"  ACM  Symposium  on  Operating  System  Principles, 
October  1967,  7p. 

L.  G.  Roberts  and  B.  D.  Wessler,  "Computer  Network  Development  to 
Achieve  Resource  Sharing,"  Proceedings  SJCC  1970,  p.543-549. 

L.  G.  Roberts,  "A  Forward  Look,"  Signal,  August  1971,  p. 77-81 . 

L.  G.  Roberts,  "ARPA  Network  Implications,"  EDUCOM  Bulletin,  Fall 
1971,  p.4-8. 

L.  G.  Roberts,  "ARPANET:  Current  Status,  Future  Plans,"  EDUCOM 
Spring  Conference,  April  1972,  p.7-12. 

L.  G.  Roberts,  "Network  Rationale:  A  5-Year  Reevaluation," 
Proceedings  C0MPC0N  1973,  February  1973,  p.3-6. 

L.  G.  Roberts,  "Dynamic  Allocation  of  Satellite  Capacity  through 
Packet  Reservation,"  Proceedings  NCC&E  1973,  p. 71 1-716. 

L.  G.  Roberts,  "The  ARPA  Net,"  in  Computer-Communication 

Networks,  N.  Abramson  and  F.  Kuo  (Eds.),  Prentice-Hall,  1973. 

S.D.  Crocker,  J.B.  Postel,  J.F.  Heafner,  R.M.  Metcalfe, 
"Function-oriented  Protocols  for  the  ARPA  Computer  Network," 
AFIPS  Conference  Proceedings  40,  May,  1972,  pp.  271-279. 


III-28 


Report  No.  4799 


Bolt  Beranek  and  Newman  Inc. 


1.4. 1.2  The  Network  Analysis  Corporation 

Led  by  Dr.  Howard  Frank,  the  Network  Analysis  Corporation 
(NAC)  of  Glen  Cove,  Long  Island,  was  put  under  contract  by  DARPA 
to  specify  the  topological  design  of  the  ARPANET  and  to  analyze 
its  cost,  performance,  and  reliability  characteristics. 

In  the  process  of  evaluating  any  of  the  parameters  of  a 
particular  network  design,  such  as  cost,  reliability,  delay,  or 
throughput,  it  is  necessary  to  simulate  the  flow  of  traffic 
through  the  proposed  network.  Then,  the  design  may  be  altered 
slightly  to  Improve  one  of  these  measures.  In  this  procedure,  it 
is  important  to  have  a  facility  for  specifying  the  routes  on 
which  traffic  will  flow  in  the  network,  and  the  procedure  must 
not  be  too  complex  since  it  must  be  repeated  so  often  in  the 
iterative  design  process.  NAC  has  developed  some  very  efficient 
methods  for  incremental  changes  to  a  shortest-path  routing 
algorithm  as  the  network  topology  is  changed.  Further,  they  have 
discovered  a  faster  shortest-path  algorithm  than  was  previously 
available,  taking  advantage  of  the  low  connectivities  usually 
present  in  most  practical  communications  networks. 

The  following  bibliography  lists  much  of  the  published  work 
by  NAC  related  to  the  ARPANET. 
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1.4. 1.3  The  Telephone  Companies 

Building  a  nationwide  communications  network  means  doing 
business  with  a  number  of  telephone  companies;  further,  if  the 
network  is  to  have  overseas  or  foreign  components,  one  must  also 
deal  with  various  international  carriers  and  national  Post 
Telephone  Telegraphs.  This  task  was  handled  by  DECCO,  which 
negotiated  with  the  relevant  telephone  companies  and  obtained  the 
specified  service  at  the  best  price.  In  the  case  of  a  circuit 
from  UCLA  to  RAND,  for  example,  most  likely  the  service  would  be 
procured  from  General  Telephone,  the  dominant  telephone  company 
in  the  Los  Angeles  area. 

In  the  example  just  given,  a  requested  circuit  fell 
completely  within  the  jurisdiction  of  a  single  telephone  company. 
To  handle  all  instances  when  the  requested  service  spanned  the 
jurisdictions  of  more  than  one  telephone  company,  the  Bell  System 
utilized  their  Long  Lines  division;  and  in  the  case  of  many  of 
the  ARPANET  circuits,  the  company  from  which  DECCO  procured  the 
service  was  Long  Lines,  which  in  turn  procured  the  components 
necessary  to  make  up  the  requested  service  from  the  regional 
telephone  companies. 

For  the  first  several  years  of  the  ARPANET  development,  the 
Long  Lines  customer  representatives  to  DARPA  were,  in  turn,  A1 
Fraser,  Bill  Gordon,  and  Ken  Stanley.  A  customer 
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representative's  job  is  to  make  the  customer  aware  of  the  kinds 
of  service  available  and  to  keep  him  happy  with  the  service  he 
receives.  Fortunately  for  the  ARPANET,  the  Bell  System 
understood  that  a  customer  building  a  nationwide  network  needed 
the  assistance  of  some  central  individual  with  a  broad  grasp  of 
the  requirements  rather  than  having  to  rely  on  the  usual 
miscellaneous  dealings  with  local  telephone  companies.  Thus  the 
Long  Lines  representative,  who  from  his  position  in  Long  Lines 
was  already  in  contact  with  his  counterparts  in  the  many  regional 
telephone  companies,  was  permitted  to  use  his  network  of  contacts 
to  provide  informal  coordination  of  the  entire  Bell  System 
service  to  the  ARPANET. 

For  instance,  installing  a  telephone  circuit  between  two 
IHPs  requires  that  the  IMPs  and  the  telephone  companies  at  both 
ends  all  be  ready  simultaneously.  It  is  useless  for  the  IMP 
supplier  and  one  of  the  telephone  companies  to  strain  to  make  a 
scheduled  date  if  the  other  telephone  company  cannot  make  that 
date.  Similarly,  it  is  useless  for  the  telephone  companies  to 
strain  to  make  a  date  if  the  IMP  supplier  cannot.  The  Bell 
System  customer  representative  took  it  upon  himself  to  coordinate 
all  such  interdependent  events.  By  virtue  of  the  Bell  System's 
willingness  to  provide  this  critical  coordination,  an  extremely 
smooth  and  efficient  relationship  has  been  built  up  between  the 
IMP  suppliers,  DARPA,  and  the  member  companies  of  the  Bell 
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System.  For  a  network  of  the  size  and  complexity  of  the  ARPANET, 
there  has  been  surprisingly  little  trouble  with  the  procurement 
and  operation  of  the  telephone  circuits. 


1 
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1.4. 1.4  Bolt  Beranek  and  Newman  Inc. 

The  contract  to  construct  the  IMP  for  the  ARPANET  was 
awarded  to  Bolt  Beranek  and  Newman  Inc.  (BBN)  of  Cambridge, 
Massachusetts,  where  it  was  carried  out  in  a  group  under  the 
leadership  of  Mr.  Frank  Heart.  Once  the  first  IMPs  were 
installed,  BBN  continued  to  play  a  central  role  in  the  evolution 
of  the  network,  operating  it  and  maintaining  it  as  well  as  doing 
the  development  necessary  for  several  major  enhancements  of  its 
capability. 

In  addition  to  Frank  Heart,  a  number  of  other  individuals  at 
BBN  have  been  involved  with  the  development  of  the  ARPANET.  The 
names  of  many  of  these  individuals  may  be  found  as  authors  of  the 
papers  on  the  ARPANET  which  have  come  out  of  BBN.  However,  one 
individual,  Robert  Kahn,  deserves  particular  mention.  After 
several  years  as  a  principal  member  of  the  group  working  on  the 
network  at  BBN,  he  moved  to  the  IPTO  office  at  DARPA  from  which 
he  has  probably  done  more  to  promote  and  support  the  continued 
advance  of  packet-switching  technology  than  any  other  individual. 

A  bibliography  of  ARPANET-related  reports  and  papers  wricten 
by  members  of  the  BBN  staff  is  included  as  Appendix  A. 
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1.4. 1.5  The  Network  Information  Center 

The  accessibility  of  distributed  resources  carries  with  it 
the  need  for  an  information  service  (either  centralized  or 
distributed)  that  enables  users  to  learn  about  those  resources. 
A  contract  was  awarded  to  the  Stanford  Research  Institute  to 
develop  and  operate  a  "Network  Information  Center"  (NIC)  to  be 
established  for  the  ARPANET.  With  the  beginning  of 
implementation  of  the  network  in  1 969 •  construction  also  began  on 
the  NIC  at  SRI. 

The  NIC  provided  several  services.  It  maintained  a  list  of 
network  participants  and  distribution  lists  for  various  special 
interest  groups  within  the  network  community.  An  archive  of 
various  document  series  was  maintained.  Documents  could  be  sent 
to  the  NIC  with  instructions  for  duplication  and  distribution  to 
the  membership  or  one  or  more  of  the  special  interest  groups.  A 
highly  structured  data  base  construction,  manipulation,  and 
display  system  (called  NLS)  was  made  available  on-line  for  use 
over  the  network*,  A  list  was  kept  of  the  resources  available  on 
hosts  throughout  the  network.  The  various  ARPANET  protocol 
specifications  were  maintained  on-line  at  the  NIC. 

The  NIC  has  had  a  hand  in  the  production  or  distribution  of 
hundreds  and  hundreds  of  documents  related  to  the  ARPANET.  A  few 
of  these  documents  describe  the  activities  of  the  NIC  itself  or 
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are  otherwise  of  special  interest  within  the 
bibliography  of  these  follows. 
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1.4. 1.6  The  Network  Measurement  Center 

One  of  the  influential  early  studies  of  communications 
networks  was  performed  by  Leonard  Kleinrock  and  is  reported  in 
his  1962  Ph.D.  thesis  at  MIT,  Communication  Nets:  Stochastic 
Message  Flow  and  Delay  (reprinted  by  Dover  Publications,  New 
York,  1964).  This  study  proved  convincingly  that  message  delays 
in  a  large  store-and-forward  network  can  be  made  very  low,  and 
thus  put  to  rest  one  of  the  early  and  persistent  objections  to 
networking.  Kleinrock  considered  several  aspects  of  the 
operation  of  communications  networks  and  their  underlying 
algorithms  in  developing  a  precise  model  for  networks.  Kleinrock 
is  today  a  faculty  member  at  UCLA. 

UCLA  was  selected  to  be  the  site  of  the  first  IMP 
installation,  to  allow  early  connection  of  an  SDS  SIGMA  7  host 
which  was  to  be  used  to  support  the  tasks  of  a  Network 
Measurement  Center  (NMC).  The  NMC  had  the  responsibility  for 
much  of  the  analysis  and  simulation  of  the  ARPANET  performance, 
as  well  as  direct  measurements  based  on  statistics  gathered  by 
the  IMP  program.  While  Kleinrock  himself  was  the  guiding  force 
at  the  NMC,  over  the  years  he  had  a  series  of  students  or  staff 
members  who  supervised  the  day-to-day  measurement  work,  including 
Gerald  Cole,  Vinton  Cerf,  Holger  Opderbeck,  and  William  Naylor 
(each  obtained  a  UCLA  Ph.D.,  although  not  necessarily  for  their 
NMC  work) . 
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'  There  has  been  a  series  of  doctoral  di-ssertations  written  by 
students  at  the  UCLA  School  of  Engineering  and  related  to  the 
work  of  the  NMC.  A  list  of  of  these  theses,  as  well  as  other 
relevant  publications  by  the  faculty  and  students  associated  with 
the  NMC,  is  included  in  the  following  bibliography. 
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1.4. 1.7  The  Network  Working  Group  and  Host  Involvement 

The  initial  design  of  the  ARPANET  as  contained  in  the  RFQ 
went  some  way  toward  specifying  certain  formats  for  inter-IHP 
communications  and  for  .^P-to-host  communication.  Less  explicit 
attention  was  given  to  host-to-host  communication,  this  area 
being  left  for  host  sites  to  work  out  among  themselves. 

To  provide  the  hosts  with  a  little  impetus  to  work  on  the 
host-to-host  problems,  DARPA  assigned  the  problem  to  Elmer 
Shapiro  of  SRI.  After  an  initial  meeting,  S.  Crocker,  S.  Carr, 
and  J.  Ruiifson  met  again  in  the  summer  and  fall  of  1968  to 
continue  discussion  of  host-to-host  protocol  issues.  Their  early 
thinking  was  at  a  very  high  level,  e.g.,  the  feasibility  of 
creating  a  portable  front-end  package  which  could  be  written  once 
and  moved  to  all  network  hosts;  a  host  desiring  to  send  data  to 
another  host  would  first  send  a  data  description  to  the  receiving 
host  which  instructed  the  front-end  package  at  the  receiving  host 
hew  to  interpret  data  coming  from  the  sending  host.  On 
Valentine’s  Day,  1969,  the  first  meeting  of  host  representatives 
and  representatives  fro*  the  NMC  and  NAC,  along  with  the  IMP 
contractor,  was  held  at  BEN  in  the  middle  of  an  enormous  snow 
storm. 


Ir.  April  1969,  a  series  of  working  notes  called  Request  for 
Comments  (RFCs)  was  established,  which  could  be  circulated  to  let 
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others  know  what  they  were  doing  and  to  obtain  the  reactions  and 
involvement  of  other  interested  parties.  They  called  themselves 
the  Network  Working  Group  (NWG). 

The  NWG  eventually  grew  quite  large,  with  representatives 
from  almost  every  host  site  in  the  network  participating,  and 
mountains  of  paper  was  circulated  describing  and  commenting  on 
various  protocols.  There  were  also  occasional  mass  meetings. 
From  about  the  time  it  was  decided  that  he  would  go  to  DARPA 
until  near  the  time  he  left  DARPA,  Stephen  Crocker  served  as 
chairman  of  the  NWG.  By  the  beginning  of  1972  the  NWG  had  grown 
too  large,  but  much  of  its  work  was  done  —  large  numbers  of 
hosts  were  communicating  over  the  network.  From  this  point 
onward,  meetings  were  limited  to  those  of  an  executive  protocol 
committee  which  net  to  discuss  general  protocol  issues  and 
provide  guidance  for  Crocker,  and  to  those  of  various 
subcommittees,  e.g.,  the  group  interested  in  the  Remote  Job  Entry 
protocol.  Even  after  the  big  meetings  stopped,  most  participant 
working  notes  were  circulated  to  most  other  participants  in  the 
network. 

Gradually  the  activities  of  the  NWG  began  to  diminish.  Many 
of  the  host  site  personnel  who  had  originally  been  active  moved 
on  to  other  tasks,  and  new  users  joining  the  network  tended  to 
use  the  defined  protocols  rather  than  becoming  involved  in  their 
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specification.  As  Crocker’s  time  for  the  NWG  group  became 
increasingly  limited,  he  appointed  A] ex  McKenzie  and  Jon  Postel 
to  serve  jointly  in  his  place.  McKenzie  and  Postel  interpreted 
their  task  to  be  one  of  codification  and  coordination  primarily, 
and  after  a  few  more  spurts  of  activity  the  protocol  definition 
process  settled  for  the  most  part  into  the  status  of  a 
maintenance  effort. 

Further  activities  of  the  Network  Working  Group  will  be 
described  in  the  section  below  on  the  development  of  host-to-host 
protocol. 
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1.4.2  Initial  Subnet  Design 

BBN*s  proposal  for  the  IMP  was  for  the  most  part  compliant 
with  the  requirements  of  tne  RFQ.  In  some  important  respects, 
however,  the  BBN  IMP  design  diverged  from  those  requirements  or 
added  constraints  that  were  not  in  the  RFQ. 

The  BBN  design  took  the  idea  of  inserting  a  communications 
processor  between  the  host  and  the  network  to  a  logical  extreme: 
it  specified  that  the  IMP  be  used  only  for  communications 
functions,  and  that  there  ?;e  maximum  logical  separation  between 
IMP  and  host.  Thus,  the  design  did  not  provide  for  IMP  repacking 
of  binary  messages  on  behalf  of  a  host  or  for  doing  character 
conversion  for  a  host.  Further,  the  design  precluded  all  host 
programming  of  the  IMP  and  any  control  of  the  IMP  by  regular 
hosts. 

The  initial  subnet  design  also  specified  minimal  control 
messages  between  an  IMP  and  its  hosts,  many  fewer  than  envisioned 
in  the  RFP.  As  the  network  later  developed,  it  proved  useful  to 
add  additional  such  control  messages. 

The  IMP  design  called  for  a  hardened  IMP,  which  would  be 
robust  in  the  face  of  physical  and  electrical  abuse.  While 
statistics  have  shown  that  hardening  did  help,  it  was  eventually 
decided  not  be  be  worth  the  added  price  of  the  hardware. 
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The  capability  to  loop  all  interfaces  wa3  included  in  the 
design.  This  has  proved  to  be  of  enormous  operational 
importance. 

A  decision  was  made  to  reload  IMPs  initially  from  paper 
tape,  and  each  IMF  was  provided  with  a  paper  tape  reader.  It  was 
always  intended  that  this  procedure  would  sooner  or  later  be 
replaced  by  loading  through  the  network,  and  it  wa3.  The  use  of 
the  paper  tape  reader  at  each  IMP  was  probably  a  useful 
simplification  in  the  beginning. 

Program  debugging  was  also  specified  to  take  place  from  a 
local  terminal,  in  the  interests  of  simplicity,  with  the 
knowledge  that  cross-network  debugging  could  be  added  later.  A 
cross-network  debugging  capability  was  added  even  before  the 
first  IMP  was  delivered. 

A  delivery  schedule  of  one  IMP  per  month  in  months  eight 
through  eleven  after  contract  start,  instead  of  all  four  at  month 
nine,  was  assumed. 

In  keeping  with  the  strict  independence  of  host  and  IMP,  the 
IMP  was  to  gather  statistics  routinely  and  transmit  them 
periodically  to  specified  hosts  rather  than  permitting  hosts  to 
control  the  IMPs'  statistics-taking  capabilities.  This  proved 
satisfactory. 
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The  initial  IMP  design  was  responsive  to  the  RFQ  in  one 
particular  way  which  was  changed  at  the  first  meeting  between 
BBN,  DARPA,  and  host  representatives.  The  RFQ  had  called  for  one 
host  per  IMP  and  six  circuits  per  IMP,  although  it  also  asked 
that  more  hosts  per  IMP  be  an  option.  Almost  immediately  it 
became  clear  that  many  host  sites  had  more  than  one  host  to 
connect  to  a  single  IMP.  Thus  before  the  first  IMP  was 
delivered,  the  design  was  changed  to  permit  two,  three,  or  four 
hosts  on  an  IMP  along  with  five,  four,  or  three  lines.  This  was 
a  simple  change  to  implement. 

In  the  area  of  the  host-to-IMP  protocol,  the  initial  IMP 
design  specified  this  protocol  as  required.  Unfortunately,  some 
aspects  of  the  host-to-IMP  protocol  had  significant  detrimental 
effect  on  the  design  and  performance  of  the  other  protocols. 
Particularly  unfortunate  have  been  the  acknowledgment  system,  the 
retransmission  system,  and  the  message  identification  system 
initially  suggested  by  the  host-to-IMP  protocol. 

It  is  crucial  for  the  IMPs  to  limit  the  rate  of  flow  of  host 
traffic  into  the  net  to  the  rate  at  which  that  traffic  is  being 
taken  out,  in  order  to  prevent  subnetwork  congestion.  The  IMPs* 
first  attempt,  which  was  insufficient,  took  the  form  of 
suggesting  that  each  message  in  a  conversation  should  be  held  by 
the  sending  host  until  an  end-to-end  acknowledgment  for  the 
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previous  message  was  received.  This  suggestion  was  adopted  as 
part  of  host-to-host  protocol  upon  which  all  the  higher  standard 
protocols  are  based.  As  a  consequence,  the  bandwidth  of  a  single 
host-to-host  protocol  connection  is  severely  limited;  given  the 
ARPANET  response  time  using  50  Kbs  lines,  waiting  for  a 
destination-to-source  acknowledgment  between  messages  typically 
limits  connection  bandwidth  to  about  10  Kbs,  in  contrast  to  the 
40  Kbs  possible  with  a  constant  stream  of  messages. 

It  was  originally  thought  that  the  ARPANET  would  lose  a 
message  so  seldom  that  there  was  no  point  in  hosts  ever  bothering 
with  message  retransmission.  Unfortunately,  resolving  various 
possible  lockups  has  required  the  subnetwork  to  discard  a  message 
occasionally,  and  the  topology  of  the  network  has  evolved  into 
long  series  of  machines  and  lines  that  increase  the  probability 
of  involuntary  message  loss.  However,  the  host-to-host  protocol 
followed  the  initial  thought  and  did  not  provide  for  message 
retransmission.  Given  the  realities  of  the  probability  of 
message  loss  in  the  network  and  given  the  host-to-host  protocol, 
which  is  inordinately  sensitive  to  any  abnormality,  the 
host-to-host  protocol  (and  protocols  based  on  it)  has  not  proved 
particularly  robust  although  it  has  been  reliable. 

As  part  of  the  host-to-IMP  protocol  end-to-end 
acknowledgment  system  described  above,  the  host-to-IMP  protocol 
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specified  an  8-bit  message  identification  number  and  suggested 
that  all  messages  in  a  single  conversation  carry  this  same 
identification  number;  in  fact,  messages  with  different 
identification  numbers  were  not  guaranteed  to  be  delivered  in  the 
order  sent.  Eight  bits  is  probably  insufficient  to  identify 
uniquely  (for  the  purpose  of  possibly  required  retransmissions) 
outstanding  messages  when  successive  messages  in  a  conversation 
are  sent  without  waiting  for  an  end-to-end  acknowledgment.  Use 
of  the  small  8-bit  message  identifier  was  one  of  the  factors  that 
prevented  reliable  high-bandwidth  connections. 

The  IMP/host  protocol  has  been  changed  so  that  it  is  no 
longer  necessary  to  wait  for  the  end-to-end  acknowledgment; 
message  order  is  now  preserved  except  for  priority 
considerations;  cases  requiring  message  retransmission  are 
unambiguously  reported  to  the  sending  host;  and  the  message 
identifier  has  been  expanded  to  a  sufficient  size. 
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1.4.3  Subnet  Development 

The  first  four  IMPs  were  developed  and  installed  on  schedule 
by  the  end  of  1969.  No  sooner  were  these  IMPs  in  the  field  than 
it  became  clear  that  some  provision  was  needed  to  connect  hosts 
relatively  distant  from  an  IMP  (i.e.,  up  to  2000  feet  instead  of 
the  expected  50  feet).  Thus  in  early  1970  a  "distant"  IMP/host 
interface  was  developed.  Augmented  simply  by  heftier  line 
drivers,  these  distant  interfaces  made  clear  that  error  control 
was  needed  on  the  host/IMP  interface.  Previously,  it  had  been 
assumed  there  would  be  no  errors  on  such  a  local  hard  wired 
connection. 

By  mid-year  of  1970,  a  series  of  network  performance  tests 
were  being  carried  out.  These  uncovered  some  flaws  which  were 
quickly  corrected,  and  some  problems  which  looked  more  worrisome. 
Also  by  mid-year,  a  rudimentary  version  of  the  network  control 
center  was  established  at  BBN. 


As  the  year 

wore 

on,  sites  continued 

to 

be 

added  to 

the 

network,  the 

IMP 

program  continued 

to 

be 

improved, 

NCC 

development  continued,  the  first  230.4  Kb  circuit  was  tested 
between  two  IMPs,  and  design  for  a  version  of  the  IMP  able  to 
support  direct  terminal  connection  was  begun.  The  latter  was 
called  the  Terminal  IMP  (TIP). 


III-53 


Report  No.  4799 


Bolt  Beranek  and  Newman  Inc. 


In  1970  major  problems  with  the  IMP  flow  control  and  storage 
allocation  techniques  were  demonstrated.  It  is  interesting  that 
even  after  these  problems  were  demonstrated  (and  they  were 
serious  enough  to  completely  halt  network  operation  under  certain 
circumstances ) *  the  network  continued  to  give  adequate  service 
for  many,  many  months  while  improvements  were  designed  and 
implemented.  The  hosts  were  simply  asked  to  not  use  the  network 
in  the  way  that  caused  the  subnetwork  problems,  and  the  hosts  did 
as  they  were  asked. 

About  three-quarters  of  the  way  through  1 97 1  the  first  two 
TIPs  were  delivered,  providing  ARPANET  access  for  the  first  time 
to  users  without  their  own  hosts  or  access  to  terminals  on  some 
other  organization’s  host. 

By  the  beginning  of  1972  it  was  recognized  that  even  the 
distant  version  of  the  IMP/host  interface  was  not  sufficient,  and 
design  for  a  IMP/host  interface  for  use  over  communications 
circuits  wa3  begun.  The  evolution  of  the  IMP/host  interface  is 
worth  a  little  additional  comment.  The  initial  bit-serial, 
asynchronous,  non-error-controlled  IMP/host  interface  was 
essentially  specified  in  the  RFQ,  in  an  effort  to  simplify 
network  connection  for  the  hosts.  This  non-standard  interface 
may  have  been  of  some  benefit  in  simplifying  the  host  connection. 
However,  its  greatest  virtue  was  the  separation  it  put  between 
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the  IMP  and  the  host.  The  IMP  and  hast  did  not  have  to  worry 
about  each  other's  word  size,  and  they  did  not  have  to  worry 
about  each  other's  timing  constraints.  It  seems  likely  that 
having  to  worry  about  these  issues  would  have  delayed  network 
operation.  However,  this  interface  also  resulted  in  a 
hodge-podge  of  interface  variations,  each  designed  for  more 
distant  operation  than  its  predecessors,  and  none  except  the 
first  was  very  elegant.  For  any  new  network,  which  need  not  fear 
the  now  proven  packet-switching  technology,  it  would  clearly  be 
better  to  use  an  industry  standard  communications  interface, 
e.g.,  HDLC,  for  every  IMP/host  connection. 

In  the  first  half  of  1972  the  TIP's  capability  was  expanded 
to  support  TIP-to-TIP  magnetic  tape  transfers.  While  this  option 
was  successfully  used  between  two  network  sites,  it  was  never 
very  elegant.  Also,  a  massive  change  in  the  IMP  software  was 
undertaken  tc  correct  the  previously  discovered  flow  control  and 
storage  allocation  problems.  In  the  second  half  of  the  year,  the 
new  version  of  the  IMP  program  was  released  in  many  small 
increments,  and  the  design  of  a  new,  ten  times  more  powerful  IMP 
was  begun. 

The  beginning  of  1973  brought  the  first  satellite  link  in 
the  network,  from  California  to  Hawaii.  Also,  with  network 
traffic  rapidly  increasing,  a  number  of  subnet  reliability 
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problems  developed  which  had  to  be  corrected.  By  mid-year,  a 
pair  of  TIPs  had  been  shipped  to  Europe,  for  use  in  Norway  and 
London.  These  brought  numerous  operational  problems.  For  the 
first  time,  circuits  had  to  be  obtained  from  a  foreign  PTT,  the 
circuits  were  relatively  slow  at  9.6  Kb,  and  like  Hawaii,  these 
TIPs  were  on  a  long  spur  of**  the  network  rather  than  being  doubly 
connected  as  IMPs  typically  are.  During  1973,  nodes  continued  to 
be  delivered,  but  there  began  to  be  a  low  level  of  switching  of 
node  locations,  to  optimize  the  use  of  various  IKP  configurations 
and  as  sites  came  on  and  went  off  the  network.  Certain 
improvements  were  also  made  to  correct  problems  with  the  routing 
algorithm.  As  1973  ended  the  first  very  distant  host3  were 
connected  to  the  network  over  telephone  lines. 

In  1974  there  were  major  efforts  to  make  the  network  more 
operationally  usable.  Subnetwork  reliability  was  improved  as  was 
TIP-to-host  communication  reliability.  Methods  for  providing  TIP 
access  control  and  accounting  and  partitioning  of  logical 
subnetworks  of  hosts  were  developed.  Methods  were  developed  to 
selectively  reload  sections  of  IMP  memory. 

In  1975  network  development  slowed  up  and  the  network  took 
on  more  and  more  of  an  operational  appearance.  Major  network 
developments  in  1975  included  delivery  of  the  first  Pluribus  IMP, 
modification  of  the  IMP  and  TIP  software  to  support  more  than 
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sixty-three  IMPs  in  the  network  and  attachment  of  the  first  two 
Satellite  IMPs  to  the  network.  By  the  end  of  1975  the  network 
was  under  DCA  management. 

Looking  back,  the  subnet  development  between  1969  and  1975 
appears  relatively  smooth,  although  there  were  many  times  during 
that  period  when  those  intimately  involved  felt  they  were  trying 
to  solve  one  crisis  or  another.  The  network  grew  slowly  enough, 
and  the  basic  technology  and  implementation  was  flexible  and 
robust  enough,  that  many  problems,  both  major  and  minor,  which 
naturally  cropped  up  with  this  new  development  were  for  the  most 
part  corrected  before  they  obstructed  the  work  of  too  many  users. 
The  fact  that  the  network  was  also  part  of  an  experiment  no  doubt 
also  made  users  more  tolerant. 


III-57 


Report  No.  4799 


Bolt  Beranek  and  Newman  Inc 


1.4.4  Host  Protocol  Development 

Specifications,  generally  called  the  IMP-to-host  protocol, 
exist  for  the  physical  and  logical  message  transfer  between  a 
host  and  its  IMP.  This  protocol  is  not  sufficient  by  itself, 
however,  to  specify  the  methods  of  communication  between 
processes  running  in  two  possibly  dissimilar  hosts.  Rather,  the 
processes  must  have  some  agreement  as  to  the  method  of  initiating 
communication,  the  interpretation  of  transmitted  data,  and  so 
forth.  Altnough  it  would  be  possible  for  such  agreements  to  be 
reached  by  each  pair  of  hosts  (or  processes)  interested  in 
communication,  a  more  general  arrangement  is  desirable  in  order 
to  minimize  the  amount  of  implementation  necessary  for 
network-wide  communication.  Accordingly,  the  host  organizations 
formed  a  group  (the  Network  Working  Group  or  NWG,  introduced 
above)  to  facilitate  an  exchange  of  ideas  and  to  formulate 
additional  specifications  for  host-to-host  communications. 

The  NWG  adopted  a  "layered"  approach  to  the  specification  of 
communications  protocols,  wherein  the  higher  layers  of  protocol 
use  the  services  of  lower  layers;  the  advantages  and 
disadvantages  of  the  layered  approach  are  discussed  elsewhere  in 
this  report.  As  shown  in  Figure  3.  the  lowest  layer  is  the 
IMP-to-host  protocol.  The  next  layer  (called  the  host-to-host 
layer  in  the  figure)  specifies  methods  of  establishing 
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Figure  3  —  Layered  Relationship  of  the 
ARPANET  Protocols 

communications  paths  between  hosts,  managing  buffer  space  at  each 
end  of  a  communications  path,  etc.  Next,  the  Initial  Connection 
Protocol  or  ICP  specifies  a  standard  way  for  a  remote  user  (or 
process)  to  attract  the  attention  of  a  network  host,  preparatory 
to  using  that  host.'  The  ICP  provides  the  analog  of  the  user 
pressing  the  attention  button  at  a  local  terminal  on  a  host.  In 
the  next  layer  is  the  Telecommunications  Network  or  TELNET 
protocol,  which  was  designed  to  support  terminal  access  to  remote 
hosts.  TELNET  is  a  specification  for  a  network  standard  terminal 
and  the  protocol  for  communicating  between  this  standard  terminal 
and  a  host.  The  next  logical  protocol  layer  consists  of  function 
oriented  protocols,  two  of  which,  File  Transfer  Protocol  (FTP) 
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and  Remote  Job  Entry  protocol  (RJE),  are  shown  in  the  figure. 
Finally,  at  any  point  in  the  layering  process,  it  is  possible  to 
superimpose  ad  hoc  protocols. 

In  the  following  subsections  we  discuss  in  some  detail  the 
events  in  the  evolution  of  the  host-to-host  and  TELNET  protocols, 
and  the  events  in  the  evolution  of  a  number  of  other  protocols  in 
somewhat  less  detail. 
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1.4. 4.1  Hont-tc-Host  Protocol 

The  Network  Working  Group  was  established  in  early  1969.  By 
December  1969  an  initial  host-to-host  protocol  had  been  specified 
which  supported  communication  between  a  terminal  on  one  host  and 
a  process  on  another  host.  At  a  meeting  in  Salt  Lake  City  in 
December  1969,  the  initial  protocol  specification  was  described 
to  Lawrence  Roberts  of  DARPA  who  was  unhappy  with  it  because  the 
initial  plan  would  not  support  transmission  of  electronic  mail 
over  the  network.  He  instructed  the  Network  Working  Group  to  "go 
back  and  get  it  right." 

By  the  spring  of  1970  several  successive  versions  of  a 
host-to-host  protocol  had  been  developed,  and  a  relatively  formal 
meeting  of  the  NWG  was  held  at  UCLA  before  mid-year  at  which  the 
latest  version  of  the  protocol  was  described.  Reactions  to  the 
described  protocol  were  very  negative.  In  June  of  1970  there  was 
a  series  of  meetings  held  at  UCLA  and  Harvard  at  which  people 
from  these  two  institutions  tried  finally  to  settle  upon  a 
host-to-host  protocol  and  specify  how  it  should  be  Implemented. 
In  August  of  1970  some  of  the  more  general  (and  some  thought  more 
exotic)  aspects  of  the  host-to-host  protocol  being  considered 
were  ordered  dropped  from  the  protocol  by  Barry  Wessler  of  DARPA, 
thus  administratively  clearing  away  some  of  those  issues  which 
had  prevented  agreement.  Tha  NWG  discussion  continued  at  the 
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1970  Spring  Joint  Computer  Conference;  in  particular,  there  was 
discussion  between  Crocker  and  Roberts  regarding  the  formality  to 
be  sought  for  the  protocol,  and  DARPA  approvals  required,  and  so 
forth.  Another  NWG  meeting  was  held  at  the  Fall  Joint  Computer 
Conference  in  November  1970  in  Houston,  Texas. 

At  a  NWG  meeting  held  in  mid-February  1971  at  the  University 
of  Illinois,  a  subcommittee  was  appointed  to  look  at  the 
host-to-host  protocol  to  see  what  changes  were  immediately 
desirable  or  necessary.  This  subcommittee  went  directly  from 
Illinois  to  Cambridge,  Massachusetts,  where  it  met  for  two  days, 
wrote  an  interim  report,  and  then  reconvened  a  month  later  in  Los 
Angeles.  It  appears  that  with  the  efforts  of  this  committee 
(known  as  the  "host-to-host  protocol  glitch  cleaning  committee") 
the  design  of  the  ARPANET  host-to-host  protocol  was  finally 
coming  close  to  being  settled. 

At  about  this  same  time  DARPA  was  beginning  to  exert  great 
pressure  not  only  to  get  the  host-to-host  protocol  settled  but 
also  to  get  it  implemented  by  the  hosts.  At  a  NWG  meeting  at  the 
Spring  Joint  Computer  Conference  in  Atlantic  City  in  May  1971, 
Alex  McKenzie  took  on  the  task  of  writing  a  definitive 
specification  of  the  host-to-host  protocol  —  not  to  invent  new 
protocol,  but  to  write  down  what  had  been  decided. 
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In  October  1971  the  final  big  NWG  meeting  was  held  at 
M.I.T.,  and  was  preceded  by  a  programmers'  workshop  at  which 
differences  in  implementations  were  clarified  and  eliminated.  In 
January  1972  a  McKenzie  document  describing  the  protocol  was 
published  and  the  ARPANET  host-to-host  protocol  has  remained 
essentially  unchanged  since. 


i 
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1.4. 4. 2  The  Evolution  of  TELNET 

Early  in  the  development  of  the  ARPANET  it  became  clear  that 
a  major  function  of  the  network  would  be  to  provide  remote  use  of 
interactive  systems.  To  allow  a  user  at  a  terminal  (connected  to 
his  local  host)  to  control  and  use  a  process  in  a  remote  host,  as 
if  he  were  a  local  user  of  that  remote  host,  a  special  mechanism 
was  required.  The  problems  to  be  overcome  are  legion:  for 
example,  the  typical  host  expects  its  interactive  terminals  to  be 
physically  attached  to  the  individual  ports  of  its  hardware 
terminal  scanner  rather  than  logically  attached  via  a  multiplexed 
connection  to  the  network;  a  given  host  expects  to  communicate 
only  with  terminals  with  certain  characteristics  (e.g., 
half-duplex,  line-at-a-time,  physical  echo,  EBCDIC  character  set, 
134.5  baud)  while  a  remote  user's  terminal  might  have  completely 
different  characteristics  (e.g.,  full-duplex, 
character-at-a-time,  no  character  echo,  ASCII  character  set,  300 
baud).  The  TELNET  protocol  was  an  attempt  to  provide  the  special 
mechanism  necessary  to  permit  such  communication. 

As  early  as  1969  a  few  hosts  had  been  programmed  on  an  ad 
hoc  basis  to  permit  terminal  access  from  another  host.  In  1971 
an  NWG  subcommittee  was  formed  to  consider  the  general  problem  of 
supporting  interactive  use  of  arbitrary  hosts  by  users  at 
arbitrary  remote  terminals.  There  was  great  controversy  in  the 
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committee  discussions,  focusing  on  four  issues:  character  set, 
connection  establishment,  echoing,  and  interrupt  capability.  By 
late  197?  there  was  enough  consensus  so  that  widespread 
implementation  of  an  early  version  of  the  TELNET  protocol  had 
been  accomplished. 

Despite  widespread  implementation  of  the  early  TELNET 
protocol,  its  heavy  and  effective  use  and  numerous  attempts  to 
declare  it  complete,  discussion  of  it  continued.  There  were 
several  problems  with  the  early  version: 

1.  Despite  the  attempt  to  permit  a  minimal  implementation 
well  suited  to  the  constraints  of  small  hosts,  there 
was  no  well-defined  minimal  implementation.  Even  if 
some  TELNET  feature  was  not  desired  for  a  given 
implementation,  it  had  to  be  provided  in  case  some 
other  implementation  commended  its  use. 

2.  The  control  structure  was  inadequate.  For  example, 

unless  some  exceedingly  constraining  assumptions  were 
made,  it  was  possible  for  the  two  ends  of  a  TELNET 
connection  to  loop,  commanding  each  other  to  take 
opposite  actions. 

3.  The  asymmetry  of  TELNET  connections  precluded  one  end 
from  initiating  certain  functions,  such  as  echoing 
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behavior.  This  seriously  constrained  the  use  of  TELNET 
protocol  for  character  communication  between  processes 
not  serving  terminals,  a  role  for  which  it  would 
otherwise  have  been  well  suited  and  for  which  it  was 
already  frequently  used  in  the  absence  of  any  better 
protocol . 

4.  The  issue  of  interfacing  character-at-a-time  hosts  to 
line-at-a-time  hosts  was  poorly  handled. 

By  early  1973  it  had  become  apparent  that  minor  adjustments 
to  the  early  TELNET  protocol  would  not  solve  these  problems  and 
that  some  fundamental  changes  were  needed.  A  new  subcommittee 
met  and,  with  the  previous  experience  to  guide  them,  developed 
several  fundamental  principles.  These  new  principles,  when  added 
to  the  earlier  principles  of  the  Network  Virtual  Terminal  and  the 
remote  interrupt  (synch)  mechanism,  resulted  in  a  revised  TELNET 
protocol  which  solved  most  of  the  earlier  problems  that  had 
precluded  universal  acceptance  of  the  protocol. 

There  was  such  enthusiasm  for  the  new  version  that  a 
schedule  for  "rapid"  (within  the  year)  implementation  was  laid 
out.  However,  the  implementation  of  the  new  TELNET  protocol 
proceeded  more  slowly  than  expected.  Only  in  the  past  year  have 
implementations  been  widely  available.  In  retrospect,  there  were 
several  reasons  for  the  delay  in  the  implementation:  1)  at  the 
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time  the  revised  protocol  implementation  was  scheduled, 
implementation  of  the  initial  version  had  been  completed  and  host 
system  managers  had  not  budgeted  resources  for  a  second 
implementation;  2)  about  this  time  DARPA's  research  interest  in 
the  network  was  declining  and  the  network  was  entering  a  period 
of  status  quo  operation;  3)  despite  initial  belief  that  a  clean 
method  of  phasing  over  from  the  initial  protocol  to  the  revised 
protocol  existed,  none  was  found  by  most  implementors  and 
consequently  most  chose  to  provide  a  complete  implementation  of 
the  revised  protocol  to  operate  in  parallel  with  the  initial 
protocol;  and  4)  implementation  for  the  most  prevalent  user 
host,  the  TIP,  proved  to  be  very  difficult  (because  of  the  TIP’s 
limited  memory)  and  time-consuming,  thus  implicitly  relieving 
pressure  on  the  server  hosts  to  implement  the  revised  protocol. 
The  new  TELNET  protocol  has  been  the  accepted  standard  for 
several  years,  and  it  is  widely  implemented  and  used. 
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1.4. 4. 3  The  Evolution  of  the  Other  Host  Protocols 

There  are  several  other  host  protocols  the  evolution  of 
which  should  be  briefly  mentioned. 

The  File  Transfer  Protocol  started  out  as  two  protocols,  a 
Data  Transfer  Protocol  and  a  File  Transfer  Protocol.  To 
over-simplify,  the  Data  Transfer  Protocol  was  to  specify  the 
format  of  data  being  transferred  and  the  File  Transfer  Protocol 
was  to  specify  how  it  was  transferred.  Eventually,  the  File 
Transfer  Protocol  alone  was  defined  with  a  data  portion  and  a 
control  portion.  After  the  final  push  to  specify  the  FTP, 
relatively  little  additional  work  was  done,  consisting  only  of  a 
little  effort  to  clean  up  fundamental  aspects  of  the  protocol, 
and  a  good  bit  of  work  reconciling  the  "reply  codes"  that 
different  hosts  used  to  indicate  FTP-related  events. 

Before  a  Remote  Job  Entry  protocol  could  be  defined  by  the 
NWG  as  a  whole,  UCLA's  IBM  360/91  host,  a  batch  oriented  host, 
needed  some  RJE-like  protocol  with  which  to  serve  a  few  users  who 
wanted  early  access  to  the  computing  power  of  that  particular 
host.  Thus,  led  by  the  UCLA  group,  a  protocol  called  the  Remote 
Job  Service  or  RJS  protocol  was  defined  and  implemented.  The  NWG 
eventually  got  around  to  working  on  the  problem  of  a  Remote  Job 
Entry  protocol  and  undertook  a  relatively  massive  effort  to 
define  such  a  protocol.  However,  by  the  time  the  RJE  protocol 
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definition  was  finished,  half  a  doz  t  or  so  hosts  had  already 
implemented  to  interim  RJS  protocol.  Since  these  included  most 
of  the  hosts  on  the  network  interested  in  supporting  remote 
batch,  there  was  little  incentive  for  them  to  implement  the  new 
RJE  protocol.  Thus,  today  the  RJE  protocol  is  carefully 
specified  but  to  our  knowledge  is  not  implemented  anywhere,  and 
the  RJS  protocol  prevails. 

Moving  upward  in  sophistication,  another  protocol  that  was 
the  subject  of  early  discussion  was  one  for  graphics.  Several 
versions  of  a  graphics  protocol  were  specified  but  until  recently 
there  was  never  widespread  implementation  of  any  of  them. 
Recently,  as  part  of  the  ACCAT  experiment,  an  operational 
graphics  protocol  has  been  developed. 

In  addition  to  the  host-to-host  protocol  which  was  finally 
specified  after  much  iteration,  a  number  of  alternative  protocols 
were  suggested  by  various  members  of  the  NWG.  Before  the 
host-to-host  protocol  was  settled  upon,  Richard  Kaline  and  David 
Walden  each  suggested  an  alternative  protocol.  Even  after  the 
adoption  of  the  host-to-host  protocol,  there  was  some  discussion 
of  experiments  with  a  protocol  derived  from  the  Walden 
suggestion.  More  recently,  as  part  of  the  DARPA-sponsored 
National  Software  works  project,  Robert  Thomas,  Stuart  Schaffner, 
and  their  colleagues  have  designed  and  implemented  a  host-to-host 
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protocol  known  as  MSG.  Another  protocol,  known  as  TCP,  deserves 
special  mention. 

Near  the  time  of  the  formation  of  the  International  Network 
Working  Group,  as  network  interconnection  began  to  be  of  great 
interest,  discussions  began  on  a  t-tandard  inter-network  protocol, 
particularly  one  which  would  correct  cone  of  the  shortcomings  of 
the  ARPANET  host-to-host  protocol.  At  the  AFIPS  1973  NCC  in  New 
York  City  a  meeting  was  held  at  which  certain  ideas  for  a  new 
host-to-host  protocol  were  discussed.  After  some  additional 
correspondence,  Robert  Kahn  of  DARPA  and  Vinton  Cerf ,  then  of 
Stanford,  got  together  and  designed  a  protocol  known  as  TCP. 
Other  members  of  INWG,  perhaps  not  satisfied  that  TCP  represented 
an  international  standard,  continued  developing  still  another 
host-to-host  protocol  (Cerf  also  participated  in  this  later 
effort).  TCP  quickly  became  DARPA’ s  choice  of  the  host-to-host 
protocol  to  be  used  in  situations  where  the  ARPANET  host-to-host 
protocol  was  insufficient  or  where  inter-networking  was  required. 
With  DARPA  support,  several  TCP  implementations  were  done  and  the 
protocol  has  come  into  relatively  widespread  use  within  the 
ARPANET,  and  its  use  is  still  spreading.  TCP  is  scheduled  to 
replace  the  ARPANET  host-to-host  protocol  throughout  the  net  by  1 
January  1983.  Meanwhile  the  host-to-host  protocol  that  the  rest 
of  INWG  was  working  on  was  finished,  and  documented,  just  as  the 
PTTs  and  North  American  common  carriers  submitted  the  X.25 
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standard  to  CCITT;  so  the  INWG  consensus  protocol  will  most 
likely  play  little  operational  role  in  the  ARPANET  or  elsewhere. 
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1.4.5  Network  Growth  —  A  Summary 

In  the  following  three  subsections  we  consider  tnree  aspects 
of  the  growth  of  the  network:  the  traffic  growth,  the  growth  of 
the  network  topology,  and  the  increase  in  the  number  and  type  of 
hosts  on  the  network. 

1.4. 5.1  Traffic  Growth 

In  early  1973,  Roberts  presented  a  curve  of  average  host 
internode  traffic  growth  for  the  network*  which  showed  the  level 
of  internode  network  traffic  to  be  increasing  at  a  rate  of  a 
factor  of  ten  every  ten  months.  Internode  traffic  means  traffic 
sent  from  a  host  on  one  node  to  a  host  on  a  different  node;  i.e., 
it  does  not  include  traffic  sent  between  hosts  on  the  same  node. 
Based  on  this  rapid  rate  of  growth,  Roberts  predicted  the  network 
would  run  out  of  capacity  in  nine  months.  As  shown  in  the 
following  figure,  shortly  after  Roberts’  prediction  the  rate  of 
internode  traffic  growth  decreased  sharply  to  roughly  a  factor  of 
two  every  twenty  months.  It  is  interesting  to  speculate  on  the 
reason  for  this  sharp  decrease. 


*  L.G.  Roberts,  "Network  Rationale:  A  5-Year  Reevaluation, " 

Proceedings  COMPCON  1973,  February  1973,  pp.  3-6. 
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ARPANET  HOST  INTERNODE  TRAFFIC 


Figure  4  —  Growth  in  Average  Host  Internode  Traffic 
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It  can  be  hypothesized  that  the  existing  hosts  (and  the  few 
new  hosts  that  were  added)  were  used  remotely  more  and  more  and 
network  traffic  increased  more  and  more,  until  the  hosts  (at 
least  the  popular  time-sharing  hosts)  began  to  run  out  of 
capacity;  this  made  it  pointless  for  new  remote  users  to  attempt 
to  get  service,  and  resulted,  in  turn,  in  a  leveling  off  of 
network  traffic  growth.  Therefore,  instead  of  the  network 
running  out  of  capacity  as  predicted  by  Roberts,  it  seems  that 
the  hosts  ran  out  of  capacity  while  the  network  still  has 
capacity  left. 

As  already  stated,  the  traffic  shown  in  Roberts'  curve  and 
in  the  figure  above  includes  only  internode  traffic.  There  are 
two  reasons  for  excluding  intranode  traffic.  First,  intranode 
traffic  puts  a  burden  on  only  one  node  rather  than  on  the  network 
as  a  whole.  Thus  when  Roberts,  for  instance,  was  attempting  to 
calculate  the  effects  of  host  traffic  on  network  capacity,  he 
naturally  excluded  intranode  traffic.  Second,  the  available 
intranode  traffic  statistics  include  some  amount  of  test  traffic 
being  looped  from  a  host  through  its  node  and  back  to  the  same 
host,  and  there  is  no  convenient  way  to  separate  this  looped  test 
traffic  from  actual  data  traffic  between  two  hosts  on  the  same 
node.  It  is  believed,  however,  that  there  is  actually  a 
significant  amount  of  real  traffic  between  hosts  on  the  same 
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node.  For  instance,  Kleinrock  reports*  that  during  a  week-long 
measurement,  the  level  of  intranode  tr'ffic  amounted  to  a  daily 
average  of  twenty  percent  of  the  level  of  internode  traffic,  and 
in  some  one-hour  intervals  the  intranode  traffic  level  was  as 
much  as  eighty  percent  of  the  internode  traffic  level.  A  scan  of 
available  long  term  statistics  on  inter-  and  intranode  traffic 
shows  that  intranode  traffic  levels  have  averaged  between  twenty 
and  forty  percent  of  internode  traffic  levels.  Thus  the  traffic 
curve  given  in  the  figure  above  should  be  scaled  up  by  this 
factor  if  all  traffic  is  to  be  included. 

That  intranode  traffic  is  a  significant  portion  of  all 
network  traffic  is  interesting  and  probably  indicative  of  four 
phenomena.  First,  the  IMP  is  a  handy  interhost  interface,  and 
once  one  is  installed  in  a  computer  center  to  connect  some  host 
onto  the  network,  there  is  very  soon  pressure  to  connect  other 
computers  in  the  computer  center  to  the  IMP  so  that  desired 
communication  between  the  computers  is  possible.  Second,  when 
two  computers  are  connected  to  the  same  IMP  so  they  may  both 
communicate  with  other  computers  in  the  network,  communication 
between  the  two  computers  themselves  comes  free  and  begins  to 
happen  even  if  it  was  not  initially  thought  to  be  desired. 


*  L.  Kleinrock  and  W.  Naylor,  "On  Measured  Behavior  of  the  ARPA 
Network,"  AFIPS  Conference  Proceedings,  Volume  43,  May  1974,  pp. 
76t-j80. 
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Third,  the  TIP  (a  host)  has  been  chosen  by  several  sites  as  the 
most  flexible  available  terminal  multiplexor  and  TIP-to-host 
traffic  at  these  sites  is  likely  to  be  intranode.  Fourth,  there 
is  a  (as  yet  still  weak,  but  definite)  tendency  for  hosts  to  be 
concentrated  at  a  certain  site  and  therefore  often  on  the  same 
IMP.  The  reason  for  this  tendency  is  that,  while  some  cynics 
would  have  guessed  that  every  computer  center  manager  is  trying 
to  build  his  empire  as  large  as  possible,  in  fact  the  world  of 
computer  center  managers  appears  to  include  not  only  managers 
whose  inclinations  are  as  the  cynics  guessed  but  also  many  who 
dislike  running  computer  centers  but  do  so  because  they  need  the 
service  supplied  by  the  computer  center.  Once  the  network  became 
available,  some  sites  have  arranged  with  some  other  sites  that 
one  site's  computer  was  moved  to  a  second  site,  and  the  second 
site  managed  it  for  the  first  site  which  used  the  computer  over 
the  network  via  a  simple  terminal  concentrator.  A  further  reason 
for  this  tendency  (for  hosts  to  be  clustered)  is  the  economy  of 
scale  possible  when  only  one  facility  and  staff  is  required  for 
the  operation  of  several  computers. 

1.4. 5. 2  Topology 

The  first  ARPANET  node  was  installed  at  the  University  of 
California  at  Los  Angeles  in  late  1969  and  the  next  three  nodes 
were  Installed  in  California  and  Utah. 
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By  June  1970  three  East  Coast  and  two  more  West  Coast  nodes 
were  added,  as  well  as  two  cross-country  lines. 
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IMPs  continued  to  be  delivered  to  the  field  at  an  average 
rate  of  approximately  one  per  month,  so  that  by  late  1970  there 
were  thirteen  IMPs  installed  in  the  network.  The  IMPs  were  all 
entirely  compatible,  all  being  based  on  the  Honeywell  516 
computer. 


Figure  7:  December  1970 
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By  two-thirds  of  the  way  through  1971,  two  additional  516 
IHPs  Had  been  installed,  the  prototype  TIP  was  running  at  BBN, 
and  two  TIPs  were  operational  within  the  network,  at  MITRE  and 
AMES. 


Figure  8:  September  1971 
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The  TIPs  were  based  on  Honeywell  316  instead  of  516 
computers  and  had  as  a  component  a  316-based  IMP  which  was 
completely  compatible  with  the  516  IMP  but  half  as  expensive.  By 
early  1972  several  additional  IMPs  and  TIPs  had  been  installed 
and  the  central  part  of  the  network  between  the  East  and  West 
Coast  clusters  was  beginning  to  fill  out. 


Figure  9:  March  1972 
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By  August  1972  a  third  cross-country  line  had  been  added  and 
it  was  clear  that  in  addition  to  the  IMPs  scattered  throughout 
the  center  of  the  country,  there  were  actually  clusters  of  IMPs 
in  four  geographic  areas,  Boston,  Washington,  D.C.,  San 
Francisco,  and  Los  Angeles. 


Figure  10:  August  1972 
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There  follow  once-yearly  maps  for  the  years  1973  to  1977 
with  which  the  reader  can  follow  the  continued  growth  of  the 
ARPANET  topology. 


Figure  11:  September  1973 
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Figure  13:  July  1975 
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Figure  14:  July  1976 
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Having  shown  the  growth  of  the  ARPANET  topology  through  a 
series  of  geographic  maps,  it  is  interesting  to  consider  the 
evolution  of  the  topology  on  a  more  quantitative  basis.  We  use 
the  topologies  given  in  the  eleven  maps  shown  above  as  the  basis 
for  the  quantitative  data  shown  in  the  Figure  16  chart.  Columns 
6  and  7  are  missing  the  first  six  entries  because  the  HAWAII, 
NORSAR,  and  LONDON  nodes  did  not  exist  at  the  time3  the  six 
earlier  maps  were  made.  The  first  three  entries  in  columns  8 
through  11  are  missing  because  that  information  was  not  kept  in 
the  early  days  of  the  network.  There  are  several  interesting 
facts  that  should  be  noted  from  the  chart.  First,  the  number  of 
network  nodes  has  stayed  about  the  same  since  1975.  Despite 
this,  the  network  throughput  has  continued  to  increase.  Next, 
note  that  intranode  throughput  actually  is  a  significant  fraction 
of  total  network  throughput.  Also,  note  the  peak  in  average  path 
length  reached  in  1974-75,  and  the  subsequent  decrease  in  average 
path  length;  selected  lines  were  added  to  the  network  in  1975-76 
as  a  direct  response  to  network  delay  problems  which  occurred  in 
1974-75.  Finally,  note  that  in  July  1975  and  July  1977*  the  path 
from  HAWAII  to  LONDON  is  equaled  by  other  paths  in  the  network; 
the  NCC  (in  the  Boston  area)  was  15  hops  from  both  UCSB  (Santa 
Barbara,  California)  and  FNWC  (Monterey,  California)  in  1975,  and 
in  1977  SRI2  (near  San  Francisco,  California)  is  eleven  hops  from 
NRL  (near  Washington,  D.C.). 
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where  the  columns  contain  the  following  information: 

Column  1 :  Date  of  Map 
Column  2:  Number  of  Nodes 
Column  3:  Average  Connectivity 
Column  4:  Average  Path  Length 

Column  Maximum  Path  Length 

Column  6:  Average  Path  Length,  Minus  HAWAII,  NORSAR,  LONDON 

Column  7:  Maximum  Path  Length,  Minus  HAWAII,  NORSAR,  LONDON 

Column  8:  Percentage  of  Node  Unavailability 
Column  9:  Internode  Throughput 
Column  10:  Intranode  Throughput 

Column  11:  Sura  of  Internode  and  Intranode  Throughput 


Figure  16:  Some  Quantitative  Data 
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There  has  been  a  good  deal  of  actual  measurement  of  the 
behavior  of  the  ARPANET,  and  the  most  detailed  discussion  of  this 
is  presented  in  the  paper  entitled  "On  Measured  Behavior  of  the 
ARPA  Network"  (L.  Kleinrock  and  W.E.  Naylor,  AFIPS  1975 
Conference  Proceedings,  Vol.  43,  pp.  767-780). 
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1 .4 .5 .3  Hosts 

The  first  four  hosts  connected  to  the  ARPANET  were  an  SDS 
SIGMA-7  at  UCLA,  an  SDS-940  at  SRI,  an  IBM  360/75  at  UCSB,  and  a 
DEC  PDP-10  at  the  University  of  Utah.  This  beginning  gave  a  go~d 
indication  of  the  diversity  of  host  manufacturer  types  and 
operating  systems  which  would  use  the  ARPANET.  There  follows  a 
recent*  schematic  map  of  the  ARPANET  (Figure  17 )  showing  the 
various  hosts.  Notice  that  the  hosts  range  from  small  PDP-lls  to 
large  IBM  systems,  with  a  scattering  of  very  special  hosts  such 
as  ILLIAC-IV ,  running  a  variety  of  operating  systems  from 
relatively  standard  ones  provided  by  manufacturers  to  very 
special  ones  constructed  by  university  researchers. 

Figure  18  provides  a  breakdown  of  the  numbers  and  kinds  of 
hosts  on  the  network.  It  is  based  on  information  compiled  by  the 
NIC  for  inclusion  in  the  ARPANET  directory.  Some  of  these  hosts 
share  a  single  host  port  on  an  IMP.  Also,  each  of  the 
twenty-three  TIPs  in  the  ARPANET  logically  provides  a  host 
function  although  no  physical  host  separate  from  the  network  node 
is  required. 


*  A  sampling  of  such  schematic  maps  covering  the  entire  history 
of  the  ARPANET  is  provided  in  Appendix  B. 
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Figure  17  —  ARPANET  Logical  Map  —  June  1977 
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Figure  18  —  Summary  of  ARPANET  Hosts 


Report  No.  4799 


Bolt  Beranek  and  Newman  Inc 


f 


I 


Report  No.  47 99 


Bolt  Beranek  and  Newman  Inc. 


1.5  The  Impact  of  the  ARPANET 

The  ARPANET  has  served  well  its  original  function  as  a 
testbed  for  a  new  computer  communications  technology.  More 
recently  the  ARPANET  has  also  given  good  operational  service  to  a 
number  of  users  who  have  come  to  depend  on  it  for  their  computer 
communications  service.  However,  a  part  of  the  original  program 
plan  for  the  ARPANET  was  technology  transfer.  For  instance,  it 
was  thought  that  the  transfer  of  interactive  computer  network 
technology  would  occur  in  three  forms:  (1)  Dissemination  cf 
techniques  and  experimental  results  through  the  open  scientific 
and  technical  literature,  (2)  Through  the  common  carriers  or 
other  commercial  organizations  concerned  with  data  transfer  and 
dissemination,  and  (3)  Through  the  military  command  and  control 
centers  for  which  the  national  Military  Command  System  Support 
Center  in  the  Pentagon  serves  as  the  focal  point.  The  ARPANET'S 
greatest  success  has  perhaps  been  in  this  area  of  technology 
transfer. 

Being  an  unclassified  effort,  implemented  for  the  most  part 
by  individuals  with  an  academic  or  research  leaning,  there  have 
naturally  been  numerous  papers  written  on  the  ARPANET.  Two  key 
sets  of  papers  written  relatively  early  in  the  ARPANET 
development  were  a  set  of  five  presented  in  a  session  at  the 
AFIPS  1970  Spring  Joint  Computer  Conference  and  another  set  of 
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five  presented  in  a  session  at  the  AFIPS  1972  Spring  Joint 
Computer  Conference.  Both  these  sessions  were  organized  by  IPTO 
and  the  two  sets  of  five  papers  were  specially  bound  together 
under  DARPA-provided  covers  and  distributed  widely.  We  have 
already  listed  these  and  a  number  of  other  papers  in  the 
bibliographies  provided  elsewhei e  in  this  history.  In  1975  IPTO 
commissioned  Becker  and  Hayes,  Inc  ,  of  Los  Angeles,  California, 
to  prepare  a  bibliography  of  publications  related  to  the  ARPANET 


Becker  and  Hayes,  Inc.,  February  1976,  I85p.).  This  bibliography 
lists  561  relevant  documents  and  includes  a  subject  index.  The 
document  is  available  from  NTIS  under  accession  number 
AD-A026900.  A  complete  set  of  the  papers  in  the  bibliography  was 
also  collected  on  microfiche.  There  was  also  a  large  number  of 
informal  working  papers  distributed  among  the  various  groups  and 
individuals  working  on  the  ARPANET  development.  Many  of  these 
are  also  covered  in  the  Becker  and  Hayes  bibliography.  The 
National  Bureau  of  Standards  has  also  constructed  a  bibliography 
on  computer  communications  which  includes  hundreds  of 
^RPANET-related  publications. 

In  mid  1971 ,  Robert  Kahn,  then  at  BBN,  suggested  the 
possibility  of  a  public  demonstration  of  the  ARPANET  at  the  1972 
Spring  Joint  Computer  Conference.  Lawrence  Roberts  arranged 
instead  for  this  demonstration  to  be  held  in  October  1972  at  the 
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first  International  Conference  on  Computer  Communication  (ICCC). 
This  demonstration  marked  a  key  turning  point  in  ARPANET 
development.  It  forced  all  participants  in  the  project  to 
thoroughly  debug  and  test  their  network  support  and  application 
protocols.  It  gave  international  visibility  to  the  packet 
switching  concept  which,  until  then,  had  been  viewed  largely  with 
scepticism  by  the  communications  community.  Robert  Kahn 
undertook  the  task  of  marshalling  resources  and  personnel  and 
with  several  key  members  of  the  ARPANET  community  planned  and 
managed  the  full  year  effort  which  culminated  in  a  successful 
demonstration.  Fifty  kilobit  phone  lines  were  leased  from 
existing  network  sites  to  the  conference  site  at  the  Washington 
Hilton  Hotel,  and  an  ARPANET  TIP  was  set  up  in  a  demonstration 
room  in  the  hotel  for  the  duration  of  the  conference.  Dozens  of 
members  of  the  ARPANET  community  were  involved.  Manufacturers  of 
all  manner  of  computer  terminals  were  invited  to  connect  their 
terminals  to  the  demonstration  TIP.  Throughout  the  conference 
hours,  each  day  of  the  conference,  there  were  individuals 
available  to  demonstrate  the  use  of  programs  on  ARPANET  hosts 
from  the  manufacturer-provided  terminals.  A  relatively  thick 
booklet  was  written,  and  many  copies  made  available  at  the 
conference,  by  means  of  which  visitors  to  the  demonstration  area 
could  follow  "do  it  yourself"  directions  to  use  programs  on  many 
network  hosts.  A  twenty  or  thirty  minute  motion  picture  about 
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the  ARPANET  and  the  promise  of  computer  communications  was 
produced  and  shown  at  the  conference.  Coming  at  a  time  when  the 
TIP  had  not  been  available  for  a  very  long  time,  when  only  a 
limited  number  of  terminals  had  been  tried  with  the  ARPANET,  and 
when  many  hosts  had  completed  the  initial  implementation  of  the 
necessary  host  software  but  few  had  had  it  running  for  very  long, 
the  ICCC  demonstration  provided  an  important  stimulus  for  the 
ARPANET  community  to  pull  together  and  get  the  network  in  true 
operational  shape.  The  demonstration  itself  was  a  spectacular 
success;  with  everything  working  amazingly  well,  many  visitors 
remarked  that  the  ARPANET  technology  "really  is  real"  and  carried 
this  impression  back  home  with  them.  The  assurance  with  which 
Roberts  promised  the  demonstration  and  the  routine  way  in  which 
he  spoke  of  it  while  it  was  happening  no  doubt  enhanced  the 
impression  taken  home  by  the  visitors,  and  belied  the  crash 
efforts  and  feelings  of  panic  of  the  members  of  the  ARPANET 
community  who  were  called  upon  to  execute  the  demonstration. 

There  have  been  numerous  other  demonstrations  of  the 
technology.  Of  course,  many  of  these  have  been  informal  or 
relatively  low  key  or  relatively  short,  but  in  several  instances 
the  demonstrations  have  been  truly  major  productions.  Once  for 
example,  IPTO,  with  help  from  several  members  of  the  ARPANET 
community,  took  a  demonstration  team  to  Europe  where  a  series  of 
demonstrations  was  given  for  members  of  the  NATO  community  over  a 
two-week  period. 
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There  has  been  good  success  in  transferring  the  ARPANET 
technology  to  other  parts  of  the  Department  of  Defense.  The 
Defense  Communications  Agency  procured  two  small  networks, 
essentially  identical  to  the  ARPANET  in  function,  for  the  purpose 
of  gaining  experience  with  the  ARPANET  technology.  Called  the 
PWIN  and  EDCN  networks,  the  first  of  these  was  used  in  the  WWMCCS 
effort  and  the.  second  was  used  in  connection  with  the  successor 
to  AUTODIN  I.  Two  other  networks  essentially  identical  to  the 
ARPANET  but  smaller  have  also  been  procured  by  other  parts  of 
DoD. 


Where  direct  copies  of  the  ARPANET  were  not  appropriate,  the 
ARPANET  technology  has  nonetheless  affected  the  characteristics 
of  new  DoD  networks  being  built.  For  instance,  the  new  DoD 
common  user  network,  AUTODIN  II,  being  constructed  by  DCA,  is 
explicitly  a  second  generation  ARPANET. 

Outside  the  U.S.  military,  the  commercial  world  has  begun  to 
use  the  ARPANET  technology  or  variations  on  it.  Several 
companies  have  filed  with  the  F.C.C.  or  already  been  licensed  to 
offer  communications  services  based  more  or  less  directly  on  the 
packet-switching  technology  developed  in  ARPANET.  Among  these 
are  Telenet  Communications  Corporation  (for  which  BBN  arranged 
the  financing  and  Lawrence  Roberts  was  President,*  Tymnet, 
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Graphnet,  IT&T,  and  AT&T.  Because  of  its  access  to  substantial 
ARPANET  expertise,  of  the  several  common  carriers  Telenet  has 
used  the  ARPANET  technology  most  directly.  Telenet  now  serves 
about  eighty  cities  in  the  continental  U.S.  and  has  made 
arrangements  to  connect  to  several  foreign  networks  and  serve 
several  foreign  cities.  Tymnet  initially  developed  in  parallel 
to  ARPANET  as  a  means  of  making  the  time-sharing  services  of 
Tymshare  available  to  a  wide  geographic  area,  and  used  a 
substantially  different  although  related  communications 
technology;  however,  a  new  version  of  Tymnet  being  built  uses 
techniques  closer  to  those  developed  for  ARPANET.  Graphnet, 
IT&T,  and  AT&T  all  have  announced  their  intention  to  provide 
public  packet-switching  services. 

A  number  of  U.S.  companies  have  also  procured  or  are 
procuring  private  corporate  networks  utilizing  many  of  the 
techniques  developed  for  ARPANET.  For  instance,  it  was  recently 
announced  that  Citibank  of  New  York  City  has  constructed  (by 
contract  to  BBN)  a  private  network  very  similar  to  the  ARPANET. 
An  increasing  number  of  commercial  RFPs  call  for  packet  switching 
or  for  functions  which  can  only  be  provided  using  packet 
switching.  A  number  of  companies  have  taken  advantage  of  the 
fact  that  the  ARPANET  technology  is  in  the  public  domain  to 
obtain  the  listings  of  the  ARPANET  software. 
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Several  PTTs  (the  foreign  national  Postal,  Telephone,  and 
Telegraph  authorities)  have  made  a  commitment  to  the  development 
of  packet-switching  networks,  there  have  been  several  foreign 
research  networks,  and  several  international  networks  are  being 
developed  or  are  under  consideration.  There  follows  a  list  of 
some  of  these  networks:* 

CIGALE  —  an  operational  network  developed  by  a  French 
government  research  agency. 

RCP  —  built  by  the  French  PTT  and  operational  as  a  testbed. 

TRANSPAC  —  under  construction  by  the  French  PTT  and 
scheduled  to  become  operational  for  public  use  in  1978. 

EPSS  —  an  experimental  packet-switching  service  built  by 
the  UK  PTT  which  became  operational  in  1976. 

CTNE  —  a  packet-switching  service  operated  by  the  Spanish 
PTT. 

Datapac  —  a  packet-switching  network  built  by  the 
Trans-Canada  Telephone  System  which  is  in  the  early  phases 
of  operation. 

*  More  detail  on  this  list  may  be  found  in  "Planned  New  Public 
Data  Networks",  P.T.  Kirstein,  Computer  Networks.  Volume  1, 
Number  2,  September  1976,  pp.  79-94;  Kirstein  is  himself  a  member 
of  the  ARPANET  community  and  was  instrumental  in  arranging  the 
installation  of  the  London  node  of  the  ARPANET. 
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JIPNET  —  practically  a  copy  of  the 'ARPANET  built  in  Japan. 

EIN  —  the  European  Information  Network,  built  jointly  by 
the  U.K.,  Switzerland,  France,  and  Italy  and  strongly 
influenced  by  CIGALE, 

EURONET  —  a  network  under  discussion  by  the  European 
Economic  C  <.mmun  i T  -  ,  initially  to  use  the  EIN  technology. 

No  doubt  them  ere  other  networks  besides  those  mentioned 
above  in  the  planning  phase  or  under  construction.  With  so  many 
networks  coming  into  being,  technology  exchange  and  standards 
become  important  issues.  From  the  time  of  the  19»72  ICCC, 
representatives  cf  various  countries  and  institutions  interested 

i 

in  computer  networks  met  informally  to  discuss  their  experience 
and  to  consider  possible  standards.  In  1972  the  International 
Network  Working  Group  (INWG)  was  formed.  Modeled  on  the  ^ARPANET 
Network  Working  Group,  Vinton  Cerf  of  the  ARPANET  community  was 
selected  to  be  INWG's  first  chairman  and  DARPA  offered  the 
services  of  the  the  ARPANET  NIC  to  coordinate  and  distribute  INWG 
working  notes.  Later  INWG  became  associated  with  the 
International  Federation  for  Information  Processing,  DARPA  cut 
back  its  NIC  support,  and  DARPA1 s  role  in  INWG  decreased.  From 
its  beginning,  INWG  was  a  forum  at  which  techniques  other  than 
those  used  in  the  ARPANET  were  considered;  nonetheless,  the 
ARPANET  for  a  long  time  remained  the  one  big,  existing  network 
against  which  new  ideas  were  compared. 
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The  international  packet-switching  standardization  effort 
has  been  especially  effective.  With  the  urging  of  Data^ac, 
Transpac,  Telenet,  and  the  UK  PTT,  CCITT  (the  international 
communications  standards  organization  at  which  all  of  the  world's 
communications  authorities  are  represented)  has  with  remarkable 
speed  adopted  standards  for  connecting  hosts  to  packet-switching 
networks  and  packet-switching  networks  to  each  other.  Known  as 
X.25  and  X.75,  these  standards  clearly  address,  for  purposes  of 
international  communication,  issues  which  were  first  seen  to  be 
of  importance  in  the  ARPANET  development,  perhaps  where  the 
ARPANET  was  seen  to  be  deficient. 

While  today  the  primary  purpose  of  the  ARPANET  is  the 
operational  transmission  of  traffic,  it  is  3till  in  the  main 
stream  of  packet-switching  network  development.  For  example, 
ARPANET  research  is  underway  to  develop: 

-  more  efficient  and  loop-free  routing  algorithms 

-  multi-destination,  broadcast,  and  group  addressing 
capabilities 

-  improved  network  flow  control  and  congestion  control 
mechanisms 

-  host  attachment  to  more  than  one  node 
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-  logical  host  naming  (i.e.,  to  permit  a  single  host  to  have 
more  than  one  logical  name) 

-  host  interface  enhancements  to  permit  internetwork 


routing 
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1 .6  Maturity  and  Handover  to  DCA 

A  memorandum  of  agreement  was  worked  out  between  DARPA  and 
DCA  for  management  of  the  ARPANET  to  be  transferred  to  DCA  as  of 
July  1  ,  1 975 »  with  a  six-month  phase-over  period  from  July  1 
until  December  31 ,  during  which  DARPA  would  continue  to  help  DCA 
with  ARPANET  management  while  DCA  acclimated  itself  to  the  job. 
The  memorandum  also  called  for  a  detailed  transition  plan  to  be 
written  which  was  completed  by  June  1975. 

Along  with  the  transfer  of  ARPANET  management  from  DARPA  to 
DCA,  the  technical  functions  that  had  been  being  performed  at  RML 
were  also  transferred  to  DCA  and  the  procurement  functions  were 
transferred  from  RML  to  DECCO,  which  is  a  field  activity  of  DCA. 

There  are  several  aspects  of  the  memorandum  of  agreement  and 
the  transition  plan  which  are  worthy  of  mention  here.  The 
network  was  to  be  an  operational  DoD  facility,  to  be  used  solely 
for  government  business.  The  concept  of  "ARPANET  sponsors"  was 
invented  with  sponsors  being  those  users  or  collections  of  users 
(e.g.,  DARPA,  NBS)  who  originally  owned  ARPANET  equipment  before 
management  of  it  was  turned  over  to  DCA.  Ownership  of  the 
equipment  was  to  remain  with  the  sponsors.  DCA  was  to  finance 
the  operation  and  maintenance  of  the  network  through  use  of  the 
DCA  managed  Communications  Industrial  Fund,  which  would  recover 
its  costs  by  a  pro-rated  allocation  to  sponsors  based  on  the 
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equipment  used  by  the  sponsors.  DCA  was  to  contract  initially 
with  BBN  and  SRI  to  perform  the  ARPANET  operations  and 
maintenance  and  NIC  functions,  and  with  NAC  initially  if 
topological  consulting  was  needed;  it  was  clearly  implied  that 
DCA  could  retain  other  contractors  to  perform  these  functions 
eventually  and  certainly  after  the  first  year.  DCA  was  to 
operate  the  network  for  a  period  of  three  years  and  thereafter  if 
necessary  until  equivalent  service  could  be  provided  on  a 
military  network. 


Report  No.  4799 


Bolt  Beranek  and  Newman  Inc. 


2.  OBSERVATIONS 

For  many  of  the  people  in  government,  at  the  major 
contractors,  and  in  the  participating  universities  and  research 
centers  the  development  of  the  ARPANET  has  been  an  exciting  time 
which  will  rank  as  a  high  point  in  their  professional  careers. 
In  1969  the  ARPANET  project  represented  a  high  risk,  potentially 
high  impact  research  effort.  The  existence  of  the  net  in 
practical  useful  form  has  not  only  provided  communications 
technology  to  meet  many  short  term  needs,  but  it  represents  a 
formidable  communications  technology  and  experience  base  on  which 
the  Defense  Department  as  well  as  the  entire  public  and  private 
sectors  will  depend  for  advanced  communications  needs.  The 
strong  and  diverse  experience  base  generated  by  the  ARPANET 
project  has  placed  this  country  ahead  of  all  others  in  advanced 
digital  communications  science  and  technology. 


III-107 


Report  Ho.  4799 


Bolt  Beranek  and  Newman  Inc 


2.1  Social  Issues 

Somewhat  expectedly,  the  network  has  facilitated  a  social 
change  in  the  United  States  computer  research  community.  It  has 
become  more  convenient  for  geographically  separated  groups  to 
perform  collaborative  research  and  development.  The  ability  to 
easily  send  files  of  text  between  geographically  remote  groups, 
the  ability  to  communicate  via  messages  quickly  and  easily,  and 
the  ability  to  collaboratively  use  powerful  editing  and  document 
production  facilities  has  changed  significantly  the  "feel"  of 
collaborative  research  with  remote  groups.  Just  as  other  major 
improvements  in  human  communication  in  the  past  have  resulted  in 
a  change  in  the  rate  of  progress,  this  social  effect  of  the 
ARPANET  may  finally  be  the  largest  single  impact  of  the  ARPANET 
development. 

A  non- trivial  question  of  considerable  importance  to  the 
country  is  whv  the  ARPANET  project  has  been  so  successful.  It 
would  certainly  be  nice  if  the  same  formula  could  be  applied  to 
other  pressing  national  needs.  While  timing,  accident,  and  luck 
must  not  be  discounted,  it  is  possible  to  identify  several 
possible  contributing  factors: 

o  Despite  the  fact  that  the  ARPANET  was  a  government 
project  and  further  despite  the  fact  that  it  was  run 
within  the  Defense  Deoartment,  it  was  possible,  at  least 
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initially,  to  free  the  research  and  development  from 
some  of  the  constraints  which  often  seriously  hamper 
other  activities.  First,  the  project  was  entirely 
unclassified.  Second,  the  provision  of  network  service 
was  for  a  very  long  time  provided  to  government 
contractors  as  a  "free  good";  this  allowed  people  to 
experiment  with  the  use  of  the  network  without  being 
forced  to  make  early  and  probably  ill-advised 
cost/benefit  decisions.  Third,  despite  a  valid 
government  and  Defense  Department  concern  about 
unauthorized  use  of  government  facilities,  it  was 
possible  to  build  the  ARPANET  without  complex 
administrative  procedures  for  access  control.  Finally, 
the  ARPANET  did  not  have  to  interconnect  directly  with 
other  existing  communication  systems;  it  was  possible  to 
explore  line  protocols  and  interface  standards  de  novo, 
in  the  best  ways  that  could  be  devised.  Thus,  the 
ARPANET  program  was  incredibly  free  of  "artificial" 
requirements  and  was  able  to  concentrate  intensively  on 
the  primary  required  research  and  development.  Much 
later  in  the  project,  after  success  had  been  assured,  it 
was  relatively  easy  to  reopen  consideration  of  some  of 
these  issues;  and  at  the  current  time,  for  example,  the 
Defense  Communications  Agency  is  charging  user  groups 
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for  network  access,  and  there  have  been  experiments 
(using  private  line  interfaces)  in  the  transmission  of 
classified  data  over  the  ARPANET,  and  research  was  done 
on  how  to  implement  network  login  procedures,  etc.,  etc. 

o  A  very  convenient  fact  was  the  common  DARPA  support  of 
both  the  "network  authority"  and  the  initial  early  group 
of  network  users.  It  was  possible  for  DARPA  to  strongly 
encourage  a  cooperative  attitude  and  cooperative 
engineering  at  the  time  in  the  project  when  such 
cooperation  was  most  critically  necessary. 

In  sum,  the  project  was  an  illustration  of  what  can  be 
accomplished  with  strong  technically  sophisticated  central 
management,  adequate  resources,  and  a  clearheaded  undeviating 
concentration  on  the  central  research  and  development  issues. 

The  largest  single  surprise  of  the  ARPANET  program  has  been 
the  incredible  popularity  and  success  of  network  mail.  There  is 
little  doubt  that  the  techniques  of  network  mail  developed  in 
connection  with  the  ARPANET  program  are  going  to  sweep  the 
country  and  drastically  change  the  techniques  used  for 
intercommunication  in  the  public  and  private  sectors.  By 
hindsight,  one  can  easily  see  the  reasons  for  this  success.  The 
primary  prior  existing  communications  techniques  (the  U.S.  postal 
service  and  the  telephone)  have  certain  serious  deficiencies: 
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The  postal  service  has  become  more  expensive  and  its  performance 
is  measured  in  days  for  delivery  of  letters.  In  the  case  of  the 
telephone,  our  increasingly  mobile  society  makes  it  difficult  to 
reach  the  desired  person  (busy  people  are  hard  to  reach  in  any 
case),  and  reaching  10  distributed  people  within  a  short  period 
of  time  is  essentially  impossible.  Leaving  telephone  messages 
works  if  a  careful  system  is  established  and  a  travelling 
individual  checks  back  with  his  answering  service  or  secretary 
frequently,  but  it  is  often  inconvenient  and  is  usually  limited 
to  the  prime  3hift  working  day  of  the  secretarial  world.  To  find 
a  busy  person  able  to  accept  a  phone  call  at  the  time  you  make  it 
is  truly  unusual,  and  some  officials  have  desks  covered  with 
little  pink  slips  reporting  on  telephone  messages  with  requests 
for  return  calls.  Into  this  milieu  was  dropped  a  technique  of 
network  mail  where  at  any  time  of  the  day  or  night  one  can  send  a 
message  to  any  number  of  other  people  and  expect  that  message  to 
semi-instantaneously  be  available  in  the  computer  mailbox  of  all 
the  recipients.  Then  one  only  has  to  assume  the  habit  on  the 
part  of  all  individuals  using  the  system  to  occasionally  check 
their  mailboxes  when  they  are  free  and  not  at  a  meeting,  and  the 
performance  of  the  communications  represents  an  immeasurable 
improvement  over  the  postal  service  or  the  telephone.  With  the 
addition  of  sophisticated  tools  for  answering  messages,  filing 
messages,  forwarding  messages,  and  categorizing  messages,  the 
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system  takes  on  yet  another  step  function  of  performance  over  the 
alternate  possibilities.  In  the  space  of  just  a  couple  of  years 
this  "computer  center  curiosity"  became  a  smashing  success  on  the 
ARPANET;  there  was  a  sufficiently  large  community  of  individuals 
who  wished  to  communicate,  and  they  all  were  relatively  mobile, 
and  the  system  overnight  became  a  way  of  life.  The  implications 
of  this  kind  of  success  are  enormous.  Perhaps  such  advances 
would  have  eventually  come  anyway,  but  there  is  no  question  that 
the  ARPANET  program  provided  a  truly  convincing  demonstration  of 
the  power  of  this  approach.  The  change  will  not  be  overnight, 
because  it  does  depend  upon  the  availability  of  terminals 
accessible  to  wider  and  wider  populations,  but  commercial  systems 
are  already  available  and  substantial  DoD  experiments  are  already 
under  way. 
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2.2  Technical  Lessons 

Leaving  the  broad  social  plane,  the  ARPANET  program  provided 
several  technical  lessons  which  are  worthy  of  general  comment: 
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2.2.1  Terminal  Handling 

Rather  early  in  the  ARPANET  program,  it  became  clear  that 
terminal  access  to  the  net  via  the  main  host  computers  was.  an 
inadequate  approach;  many  classes  of  users  required  direct 
terminal  access  to  the  net  in  order  to  use  major  hosts  at  other 
locations.  The  first  response  to  this  pressure  was  the  design  of 
the  Terminal  Interface  Message  Processor  (TIP)  by  the  ARPANET 
prime  contractor,  Bolt  Beranek  and  Newman  Inc.  The  TIP  was 
designed  to  address  a  limited  problem:  to  handle 
character-oriented  asynchronous  terminals  only  and  to  be  an 
integral  part  of  the  network  authority  and  not  available  for  user 
programming  or  special  user  features.  This  limited  goal  and 
hard-nosed  attitude  permitted  the  rather  rapid  completion  of  the 
TIP  design,  the  fielding  of  many  TIPs,  and  the  rapid  availability 
of  widespread  terminal  access  to  the  network.  Thus,  the  TIP 
effort  was  extremely  successful  in  reaching  its  limited  goal. 

Unfortunately,  but  perhaps  not  surprisingly,  the  limited 
goal  and  absolute  restriction  on  user  programming  created 
considerable  unhappiness  in  portions  of  the  potential  user 
community,  and  created  considerable  pressure  for  other  "better" 
terminal  access  techniques.  Some  of  the  complaining  was  fully 
justified;  this  approach  to  servicing  interactive  character 
terminals  had  "leapfrogged"  a  whole  segment  cf  the  industry  that 
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was  concerned  with  batch  processing  and  the  use  of  synchronous 
line  disciplines  from  such  batch  processing  units.  Other 
complaints  were  less  rational  and  more  self  serving,  where  some 
groups  really  wanted  "a  computer  of  their  own"  under  the  guise  of 
obtaining  terminal  access  to  the  ARPANET.  Still  other  criticism 
was  based  on  an  honest  difference  of  opinion  as  to  the  relative 
ease  of  designing  and  deploying  a  terminal  access  device  that 
would  permit  user  programming  and  would  handle  a  broader  class  of 
terminals.  Finally,  some  criticism  was  provided  by  highly 
sophisticated  users  who  were  used  to  the  terminal  support 
"services"  provided  by  the  most  advanced  large  hosts,  and  did  not 
like  the  limited  services  provided  by  the  tiny  mini-host  resident 
in  the  TIP. 

In  response  to  this  pressure,  DARPA  for  a  time  supported  the 
development  of  a  device  called  the  "ARPA  Network  Terminal  System" 
(ANTS).  This  was  a  system  based  on  a  PDP-11  which  was  intended 
to  provide  user  programming  ability,  the  handling  of  synchronous 
terminals  as  well  as  asynchronous  terminals,  and  a  more  powerful 
set  of  services  to  the  terminal  user.  Unfortunately,  the  goals 
were  somewhat  ambitious  and,  although  a  few  ANTS  were  put  into 
the  field,  the  configuration  management  of  the  program,  the 
difficulty  in  debugging  fielded  versions  of  the  system,  and 
delays  in  implementation  eventually  led  to  a  cessation  of  DARPA 
support  for  the  effort.  In  response  to  a  somewhat  different  set 
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of  requirements,  another  attempt  was  made  with  a  system  called 
"ELF".  Here  DARPA  support  was  provided  to  try  to  standardize, 
improve,  and  deploy  a  PDP-11  based  terminal  support  software 
system  which  had  been  independently  developed  for  a  particular 
project.  Again,  the  hope  was  to  permit  user  programming  and  the 
handling  of  a  wider  class  of  terminals.  From  the  viewpoint  of 
deployment,  the  efforts  to  use  ELF  were  much  more  successful; 
goals  were  far  more  limited,  and  more  attention  was  paid  to 
orderly  development  and  maintenance.  However,  ELF  has  probably 
been  more  useful  as  a  base  for  individualized  mini-hosts  serving 
local  specialized  host  functions  than  as  a  means  of  widely 
distributed  network  access  for  large  numbers  of  terminal  users. 
At  the  time  the  ARPANET  was  transferred  to  DCA,  primary  terminal 
access  was  still  either  through  the  main  network  hosts  or  via  the 
many  TIPs  in  the  network.  There  are  several  morals  to  this 
history.  First,  it  is  extremely  difficult  to  build  a  system 
which  can  handle  all  possible  terminals;  it  is  a  bit  like  the 
"everything"  machine  and  leads  to  an  unlimited  expansion  of  the 
problem.  Second,  very  great  attention  must  be  paid  to  software 
configuration  management,  central  program  design  and  release, 
central  maintenance  and  software  support,  and  close  control  of 
program  changes  if  an  evolving  computer-based  device  is  to  be 
deployed  in  considerable  numbers  around  the  network.  There  is 
still  an  open  technical  question  whether  such  a  device  could  be 
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sensibly  and  cost-effectively  fielded  in  a  way  which  would  permit 
both  some  level  of  user  programming  to  tailor  the  functions  of 
the  device  and  a  standard  set  of  controlled  basic  services;  at 
this  point  in  the  development  of  the  technology  a  conservative 
approach  precludes  such  user  programming  if  a  device  is  to  be 
widely  deployed. 

Another  related  aspect  of  the  terminal  handling  problem  has 
to  do  with  the  management  extent  of  the  network  authority.  It 
was  discovered  that  when  an  unsophisticated  U3er  typed  at  a 
terminal  in  a  remote  location  and  something  went  wrong  (that  is, 
the  proper  response  was  not  received),  the  user  typically  "blamed 
the  network"  despite  the  fact  that  any  number  of  possible  things: 
the  terminal  itself,  the  local  modem,  the  local  line,  the  modem 
at  the  other  end  of  the  local  line,  the  terminal  handling  device 
(e.g.,  TIP),  the  network  itself,  the  eventual  host  computer,  or 
any  similar  point  on  the  return  path,  could  have  been  at  fault. 
Thus,  it  is  extremely  important  that  the  network  authority  have 
adequate  administrative  control  over  the  entire  collection  of 
equipment  from  the  terminal  right  through  to  the  host  computers 
if  it  is  to  be  in  a  position  to  respond  to  such  unstructured 
outcries  of  rage  from  the  end  terminal  user.  During  the  ARPANET 
program,  numerous  "crisis"  incidents  were  precipitated  by  the 
failure  of  equipment  which  was  not  in  any  way  under  the  control 
of  the  network  authority.  [Perhaps  the  best  single  example  was 
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difficulties  in  the  local  on-base  telephone  system  at  Ames  where 
it  took  a  massive  effort  to  eventually  uncover  the  offending 
equipment  and  where  the  network  authority  was  forced  in  its  own 
defense  to  participate  and  lead  in  the  massive  debugging  activity 
even  though  the  offending  device  eventually  found  was  clearly 
outside  the  network  authority's  control.]  The  generalizable 
lesson  from  this  kind  of  history  is  that  devices  that  attach  to 
networks  must  be  extremely  clearly  in'  somebody  Vs  camp  of 
responsibility.  Either  the  device  must  clearly  belong  to  the 
network  authority  and  must  have  built-in  techniques  for  debugging 
and  failure  location,  or  the  device  must  clearly  belong  to  the 
host  organization  or  some  other  well  identified  group  wu\o 
understands  its  responsibilities  for  maintenance  and  trouble 
location;  devices  cannot  sit  in  cracks  between  different 
authorities.  Although  this  idea  is  really  quite  simple,  it  is 
frequently  overlooked. 

A  final  point  on  the  general  terminal  handling  problem  has 
to  do  with  the  location  of  the  "intelligence"  for  dealing  with 
terminal  related  matters.  In  general,  the  data  processing  power 
can  either  be  in  the  terminal  itself,  in  the  terminal  handling 
network  port  (e.g.,  the  TIP),  in  the  eventual  final  host  computer 
or,  of  course,  distributed  among  these  three  locations  in  some 
way.  As  the  cost  of  data  processing  is  dropping,  more  and  more 
intelligence  is  appearing  in  the  terminal  itself  and  this  entire 


III-U8 


Report  No.  4799 


Bolt  Beranek  and  Newman  Inc. 


matter  must  be  periodically  revieued  to  ensure  that  the 
performance  can  take  advantage  of  technological  change. 


system 
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2.2.2  Reliability  and  Fault  Isolation 

At  the  outset  of  the  ARPANET  design  considerable  effort  was 
expended  toward  built-in  techniques  of  fault  isolation  and 
recovery.  For  example,  IMP-to-IMP  circuits  could  be  cross 
patched  for  fault  isolation,  and  IMPs  themselves  had  power  fail 
recovery  mechanisms  and  mechanisms  for  reloading  one  IMP  from  a 
neighbor.  As  the  network  grew,  it  was  therefore  somewhat  of  a 
surprise  that  the  initial  efforts  down  this  path  were  not  nearly 
enough.  Over  the  life  of  the  ARPANET  program,  there  was  a  nearly 
continuous  effort  to  add  techniques  and  improve  existing 
techniques  for  fault  isolation,  rapid  recovery,  and  containment 
of  failures.  The  lesson  probably  is  that  it  is  difficult  to  have 
too  much  attention  to  fault  isolation  and  recovery  mechanisms. 
Many  interesting  techniques  were  slowly  added  to  improve  the 
network's  ability  to  cope  with  trouble: 

o  For  critical  pieces  of  code  such  as  the  routing 
computation,  the  code  itself  was  checksummed  before  it 
was  operated.  Similarly,  key  data  structures  were 
checksummed  before  they  were  accessed.  This  kind  of 
protection  was  added  when  it  was  discovered  that  the 
trouble  in  these  particular  classes  of  computations 
could  cause  network-wide  failures  rather  than  simply 
local  difficulties. 
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o  It  was  discovered  that  dial-in  modems  for  remote 

terminals  could  break  or  "hang"  and  there  was  no  simple 
way  to  discover  this  had  happened.  A  technique  was 
designed  wherein  a  centrally  located  autodialer 

controlled  by  the  network  authority  used  an  out  WATS 
line  to  periodically  dial  and  test  all  dial-in  ports  all 
over  the  network  (at  least  in  the  continental  United 
States) . 

o  It  was  discovered  tha*-,  when  a  difficulty  occurred  in  an 

IMP  which  caused  an  automatic  program  reload,  the 
necessary  debugging  information  was  lost  and  the  trouble 
would  likely  recur.  A  technique  was  added  to 

automatically  dump  offending  code  before  a  reload  was 
attempted,  which  then  permitted  comparison  with  a  master 
copy  and  improved  capability  for  debugging. 

o  The  ARPANET  program  was  forced  to  cope  with  the 

simultaneous  need  for  a  twenty-four  hour  a  day 
continuously  operating  reliable  system  and  at  the  same 
time  relatively  constant  levels  of  growth,  modification, 
and  change.  After,  some  early  false  starts  when  large 
changes  introduced  periods  of  substandard  performance,  a 
technique  was  evolved  whereby  greater  attention  was  paid 
to  dividing  a  large  change  into  small  incremental 
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changes  that  were  compatible  with  the  previous  system. 
Then,  when  trouble  occurred,  debugging  could  be 
concentrated  on  the  small  incremental  change  or  retreat 
taken  to  the  previous  release. 

o  As  network  users  began  to  depend  upon  a  few  particular 
"service"  hosts  for  a  wide  variety  of  services, 
including  message  services,  it  became  more  important 
^hat  such  hosts  maintain  availability  to  the  network. 
In  some  cases,  a  host  might  be  operating  adequately  as 
perceived  by  its  own  local  operators  and  yet  in  some  way 
not  be  properly  servicing  the  network  connection.  A 
technique  was  evolved  whereby  the  network  authority 
added  software  tools  to  "watch"  these  particular 
critical  hosts  and  was  then  in  a  position  to  urge 
corrective  action  by  the  local  host  authorities  if  and 
when  necessary. 
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2.2.3  Maintenance  Management 

In  the  early  years  of  the  ARPANET  program,  the  IMPs  and  TIPs 
of  the  network  were  maintained  by  subcontract  to  the  manufacturer 
of  the  basic  mini  computer  (Honeywell).  This  represented  a 
cost-effective  approach  because  Honeywell  had  maintenance 
facilities  in  many  cities.  However,  this  rather  standard  form  of 
computer  maintenance  was  simply  inadequate  for  the  high 
reliability  requirements  of  the  ARPANET.  After  a  time, 
maintenance  of  the  network  nodes  was  undertaken  directly  by  Bolt 
Beranek  and  Newman  Inc.,  the  network  prime  contractor,  and 
special  new  techniques  were  developed  which  greatly  improved  the 
average  nodal  reliability.  In  particular,  techniques  of  central 
maintenance  management  were  developed  wherein  a  very  strong  team 
of  hardware  and  software  experts  at  the  central  Network  Control 
Center  acted  in  close  concert  with  a  small  number  of  field 
personnel  who  became  highly  expert  in  the  IMP  and  TIP  machines. 
The  central  staff  could  use  the  network  itself  to  observe  the 
behavior  of  an  offending  node  and  it  could  talk  through 
difficulties  with  the  local  maintenance  engineer.  Further,  the 
people  in  the  field  became  much  more  dedicated  and  responsive  to 
ARPANET  difficulties  as  compared  to  time-shared  Honeywell 
personnel  who  had  to  take  responsibility  for  many  different  kinds 
of  equipment  in  their  geographical  territory.  This  approach  to 
maintenance  probably  has  important  benefits  for  other  distributed 
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systems,  especially  as  those  systems  will  increasingly  be 
interconnected  in  networks  and  thereby  accessible  to  central 
maintenance. 
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B.  Selection  of  ARPANET  Logical  Maps 


The  following  logical  maps  of  the  ARPANET  are  included: 


December  1970 
April  1971 
August  1971 
March  1972 
August  1972 
September  1973 
June  1974 
November  1974 
January  1975 
June  1975 
October  1975 
July  1976 
October  1976 
January  1977 
March  1977 


B-1 


ARPA  NET.  DECEMBER  1970 


1 


ARPA  NET,  AUGUST  1971 


iTit] V  >  sion mi  I 

I  (0l-d0d) 


ARPA  NETWORK,  LOGICAL  MAP,  AUGUST  1972 


ARPA  NETWORK,  LOGICAL  MAP,  JANUARY  1975 


O  IMP 
O  TIP 


mpxPA  wE  I  wORrs,  LOuiCAl  MAr,  (XiOBtrc  1,  i97b 


IMP 

TIP 

PLURIBUS  IMP 


ARPANET  LOGICAL  MAP,  OCTOBER  1976 


(PLEASE  NOTE  THAT  WHILE  THIS  MAP  SHOWS  THE  HOST  POPULATION  OF  THE  NETWORK  ACCOROING  TO  THE  BEST 
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